MSA Metadata File: msa

While it doesn’t contain any actual science data, the NIRSpec MSA metadata file is nonetheless a crucial component of calibration processing for NIRSpec MOS exposures. It contains all the slitlet, shutter, and source configuration information that’s needed by the calwebb_spec2 pipeline to process a MOS exposure. These metadata are originally generated by the MSA Planning Tool (MPT) in APT and passed through the ground system to populate the FITS metafile. Details of the metadata and how it is used in calibration processing are given below.

File names for MSA metadata files are of the form:

jw<ppppp><ooo><vvv>_<cfg>_msa.fits

where

  • ppppp: program ID number

  • ooo: observation number

  • vvv: visit number

  • cfg: MSA configuration

such as jw01180025001_01_msa.fits.

Each MOS science exposure contains a primary header keyword MSAMETFL that gives the name of the MSA metadata file to be used when processing the exposure. The values of the primary header keywords MSAMETID and PATT_NUM are also used when retrieving relevant rows of data from the tables in the MSA metadata file. If any of these keywords is missing from the input science exposure, processing will not be able to proceed and an error will be raised.

File Structure

The overall structure of the MSA FITS file is as follows:

HDU

EXTNAME

HDU Type

Data Type

Dimensions

0

N/A

primary

N/A

N/A

1

SHUTTER_IMAGE

IMAGE

float32

342 x 730

2

SHUTTER_INFO

BINTABLE

N/A

variable x 13 cols

3

SOURCE_INFO

BINTABLE

N/A

variable x 8 cols

  • PRI: The primary data array is empty. The primary header contains a small amount of metadata related to the observation and MSA configuration.

  • SHUTTER_IMAGE: 2-D data array containing a map of MSA shutters that’ve been commanded open for this MSA configuration.

  • SHUTTER_INFO: 2-D FITS binary table containing metadata for each shutter involved in this MSA configuration. One table row per shutter.

  • SOURCE_INFO: 2-D FITS binary table containing metadata from the source catalog used to create the MSA configuration. One table row per source.

The SHUTTER_INFO Metadata

The structure of the SHUTTER_INFO table extension is as follows:

Column Name

Data Type

Contents

SLITLET_ID

int16

Slitlet ID number

MSA_METADATA_ID

int16

MSA metadata ID

SHUTTER_QUADRANT

int16

MSA Quadrant number

SHUTTER_ROW

int16

Shutter row index

SHUTTER_COLUMN

int16

Shutter column index

SOURCE_ID

int32

Source ID number

BACKGROUND

string

Background flag

SHUTTER_STATE

string

Shutter state

ESTIMATED_SOURCE_IN_SHUTTER_X

float32

Source x position

ESTIMATED_SOURCE_IN_SHUTTER_Y

float32

Source y position

DITHER_POINT_INDEX

int16

Dither/nod index

PRIMARY_SOURCE

string

Primary source flag

FIXED_SLIT

string

Fixed slit name

  • SLITLET_ID: integer ID of each slitlet consisting of one or more open shutters.

  • MSA_METADATA_ID: integer ID corresponding to a particular MSA configuration or MPT plan (typically the same for all rows in a given metafile).

  • SHUTTER_QUADRANT: integer ID of the MSA quadrant in which the open shutter is located.

  • SHUTTER_ROW: integer row index of the shutter within the MSA quadrant.

  • SHUTTER_COLUMN: integrer column index of the shutter within the MSA quadrant.

  • SOURCE_ID: unique integer ID for each source in each slitlet, used for matching to an entry in the SOURCE_INFO table.

  • BACKGROUND: Y or N. Y indicates that the shutter is open to background only.

  • SHUTTER_STATE: OPEN or CLOSED. Generally will always be OPEN, unless the shutter is part of a long slit that is not contiguous.

  • ESTIMATED_SOURCE_IN_SHUTTER_X/Y: the position of the source within the shutter in relative units, where 0,0 is bottom-left, 0.5,0.5 is center, and 1,1 is upper-right, as planned in the MPT.

  • DITHER_POINT_INDEX: integer index of the nod sequence; matches to header keyword PATT_NUM.

  • PRIMARY_SOURCE: Y or N. Y indicates that the shutter contains the primary science source.

  • FIXED_SLIT: string name of a fixed slit containing the source; set to NONE for MSA slitlets. This column may not appear in older versions of the MSA metadata files.

It is the assign_wcs step in the calwebb_spec2 pipeline that opens and loads all of the necessary information from the MSA metadata file. For each slitlet that it identifies as being used in the current exposure, a Slit object is created and populated with all the shutter and source information needed in subsequent processing steps. The list of Slit objects is attached to the data model for the exposure being processed and passed along to all subsequent calibration steps.

For a given exposure, only the rows of the shutter table where the values of MSAMETID and PATT_NUM in the science exposure match the values of msa_metdata_id and dither_point_index, respectively, are loaded.

Slitlets with a catalog source

To better understand the ways in which these metadata are used, it’s useful to reference a hypothetical example of data within a SHUTTER_INFO table. The table below shows the first 9 rows of a SHUTTER_INFO table for a MOS exposure that’s part of a standard 3-point nod MOS observation, in which the sources are moved to different shutters within each slitlet from one exposure to another in the observation. Only the data for slitlet 2 are shown. In this example, slitlet 2 is comprised of 3 shutters. Because a 3-point nod pattern has been used, there are 3 different sets of metadata for each slitlet (one set for each dither/nod position) and hence a total of 9 entries (3 shutters x 3 dithers).

Slit

ID

Meta

ID

Quad

Row

Col

Src

ID

Bkg

State

X

pos

Y

pos

Dith

Pt

Pri

Src

Fxd

Slit

2

1

2

10

154

0

Y

OPEN

NaN

NaN

1

N

NONE

2

1

2

10

155

42

N

OPEN

0.399

0.702

1

Y

NONE

2

1

2

10

156

0

Y

OPEN

NaN

NaN

1

N

NONE

2

1

2

10

154

42

N

OPEN

0.410

0.710

2

Y

NONE

2

1

2

10

155

0

Y

OPEN

NaN

NaN

2

N

NONE

2

1

2

10

156

0

Y

OPEN

NaN

NaN

2

N

NONE

2

1

2

10

154

0

Y

OPEN

NaN

NaN

3

N

NONE

2

1

2

10

155

0

Y

OPEN

NaN

NaN

3

N

NONE

2

1

2

10

156

42

N

OPEN

0.389

0.718

3

Y

NONE

The values in the slitlet_id column show that we’re only looking at table rows for slitlet 2, all of which come from MSA configuration (msa_metadata_id) 1. The shutters that make up slitlet 2 are all contained in MSA quadrant 2 and are located in row 10 of that quadrant. The 3 shutters making up slitlet 2 span MSA columns 154, 155, and 156. The remaining table entries in each row vary depending on whether a source is located in a given shutter for each dither/nod position. You can see that the source is located in shutter column 155 in dither position 1, 154 in dither position 2, and 156 in dither position 3. The source_id column shows that source 42 is in those shutter positions for those dithers. The background column, meanwhile, indicates whether a given shutter only has background signal in it for each dither position.

When there is a source in a shutter, the estimated_source_in_shutter_x and estimated_source_in_shutter_y columns are populated with numerical values that estimate the source location in the shutter. These are fractional position values that run from 0,0 at the lower-left corner of a shutter to 1,1 at the upper-right corner (hence 0.5,0.5 indicates the shutter center). These source positions are used in calibration steps such as wavecorr and pathloss to compute the appropriate corrections.

The flags in the primary_source column are used to identify which shutter within each slitlet should be considered the “primary” shutter. This is especially important for slitlets that contain extended sources and hence the source_id and background entries may indicate that the source is present in multiple shutters.

Slitlets without a catalog source

It is possible for users to define slitlets that do not contain a source that’s defined in the MPT catalog when constructing an MSA configuration for an observation. These kinds of slitlets can be used for two purposes. First, slitlets in which all of the constituent shutters only contain background can be used to perform “master background” subtraction during the calwebb_spec2 pipeline processing (see NIRSpec Master Background for more details). Second, slitlets made up of open shutters that may contain signal from some uncataloged source in the field can also be created. These are referred to as “virtual” slitlets.

Background and virtual slitlets have unique metadata in the shutter information table. The primary defining piece of data is their assigned source_id value, because these slitlets don’t have a corresponding source listed in the source information table. During creation with the MPT, all background slitlets are given a source_id of zero. Virtual slitlets, on the other hand, are assigned negative source_id values, starting with -1 and counting downwards from there (i.e. each virtual slit has a unique negative source_id value).

During the parsing of shutter information described in the previous section, when a slitlet is found that has no shutters with a primary source (i.e. no shutters having primary_source = “Y”), it is recognized as a background slitlet. In order to be able to track multiple background slitlets through the remaining processing, they are reassigned a new source_id value equal to their slitlet_id. Virtual slitlets, meanwhile, retain their unique negative source_id values throughout processing.

During Stage 3 processing with the calwebb_spec3 pipeline, unique source-based product file names will be created that distinguish data from the three different kinds of slitlets: source, background, and virtual. As described in source-based file names, the SourceID field of Stage 3 file names consists of the 9-digit source_id number assigned to each MOS slitlet, preceded by one of the three characters “s”, “b”, or “v”, which identifies whether the data are from a source, background, or virtual slitlet, respectively. Note that, as described above, the source_id number used here for background slitlets is a copy of their slitlet_id number. For example, a Stage 3 file name for data taken from a virtual slitlet with source_id = -42 will look like:

jw12345-o066_v000000042_nirspec_f170lp_g235m_x1d.fits

Fixed Slits

It is possible to plan fixed slit sources alongside standard MOS targets. In this case, a unique slitlet_id is not assigned in the MSA file. Instead, the slit is identified by the value of the fixed_slit column. This value may be set to any of the NIRSpec fixed slit names used for science: S200A1, S200A2, S400A1, or S1600A1.

Fixed slit targets must always have primary_source = “Y” and background = “N”. They will never be extracted as background sources.

The shutter_quadrant, shutter_row, and shutter_column fields are set to placeholder values. All other values have the same meaning and values as for MSA slitlets.

The SOURCE_INFO Metadata

The structure of the SOURCE_INFO table extension is as follows:

Column Name

Data Type

Contents

PROGRAM

int32

Program ID number

SOURCE_ID

int32

Source ID number

SOURCE_NAME

string

Source name

ALIAS

string

Source alias

RA

float64

Source RA

DEC

float64

Source Dec

PREIMAGE_ID

string

Pre-image ID

STELLARITY

float64

Source stellarity

  • PROGRAM: 5-digit JWST program number.

  • SOURCE_ID: unique integer identifier for each source in MPT catalog.

  • SOURCE_NAME: source name, typically constructed as the concatenation of program number and source ID number.

  • ALIAS: alternate source identifier from the MPT catalog.

  • RA/Dec: catalog source coordinates, in decimal degrees.

  • PREIMAGE_ID: name of a NIRCam mosaic used to determine the source catalog, if one was used.

  • STELLARITY: DAOphot-style stellarity value for the source, where 0 is fully extended and 1 is an unresolved point source.

The table below shows an example of a few rows of hypothetical source metadata, consisting of the program ID, source ID, source name, source alias, source RA and Dec, pre-image ID, and source stellarity.

PID

Src

ID

Src

Name

Alias

RA

Dec

Pre

Image

Stellarity

1180

42

1180_0042

Bob

53.1456291

-27.7674976

95065001_001

1.00

1180

1001

1180_1001

Sue

53.1435047

-27.7689669

95065001_001

0.00

1180

3333

1180_3333

Erin

53.1485349

-27.7696165

95065001_001

0.23

1180

9876

1180_9876

Dave

53.1461433

-27.7617165

95065001_001

0.88

For each slitlet identified as having a source assigned to it in the shutter metadata, the source name, alias, RA, Dec, and stellarity are retreived from the SOURCE_INFO table and stored with the Slit object created in the calibration software. The stellarity values are used in the source type step to determine whether the source should be treated as point-like or extended.