Pipeline Stages¶

End-to-end calibration of JWST data is divided into 3 main stages of processing:

• Stage 1 consists of detector-level corrections that are performed on a group-by-group basis, followed by ramp fitting. The output of stage 1 processing is a countrate image per exposure, or per integration for some modes. Details of this pipeline can be found at:

• Stage 2 processing consists of additional instrument-level and observing-mode corrections and calibrations to produce fully calibrated exposures. The details differ for imaging and spectroscopic exposures, and there are some corrections that are unique to certain instruments or modes. Details are at:

• Stage 3 processing consists of routines that work with multiple exposures and in most cases produce some kind of combined product. There are unique pipeline modules for stage 3 processing of imaging, spectroscopic, coronagraphic, AMI, and TSO observations. Details of each are available at:

In addition, there are several pipeline modules designed for special instrument or observing modes, including:

Pipeline Classes and Configuration Files¶

Each pipeline consists of a certain sequence of calibration steps and is defined as a python class within a python code module. The pipelines can be executed from the command line either by referencing their class name or by supplying a configuration (.cfg) file that in turn references the pipeline class (see Running From the Command Line). From within python, the pipelines are called by their class names, but configuration files can still be supplied in order to set pipeline or step parameter values (see Running From Within Python). The table below lists the pipeline classes that are currently available, the corresponding configuration files that call those classes, and the observing modes for which they are intended. Note that there are some pipeline modules that are referenced by more than one configuration file.

Pipeline Class

Configuration File

Used For

Detector1Pipeline

calwebb_detector1.cfg

Stage 1: all non-TSO modes

calwebb_tso1.cfg

Stage 1: all TSO modes

DarkPipeline

calwebb_dark.cfg

Stage 1: darks

GuiderPipeline

calwebb_guider.cfg

Stage 1+2: FGS guiding modes

Image2Pipeline

calwebb_image2.cfg

Stage 2: imaging modes

calwebb_tso-image2.cfg

Stage 2: TSO imaging modes

calwebb_wfs-image2.cfg

Stage 2: WFS&C imaging

Spec2Pipeline

calwebb_spec2.cfg

Stage 2: spectroscopy modes

calwebb_tso-spec2.cfg

Stage 2: TSO spectral modes

calwebb_nrslamp-spec2.cfg

Stage 2: NIRSpec lamps

Image3Pipeline

calwebb_image3.cfg

Stage 3: imaging modes

WfsCombineStep

calwebb_wfs-image3.cfg

Stage 3: WFS&C imaging

Spec3Pipeline

calwebb_spec3.cfg

Stage 3: spectroscopy modes

Ami3Pipeline

calwebb_ami3.cfg

Stage 3: NIRISS AMI mode

Coron3Pipeline

calwebb_coron3.cfg

Stage 3: Coronagraphic mode

Tso3Pipeline

calwebb_tso3.cfg

Stage 3: TSO modes

Pipelines vs. Exposure Type¶

The data from different observing modes needs to be processed with different combinations of the pipeline stages listed above. The proper pipeline selection is usually based solely on the exposure type (EXP_TYPE keyword value). Some modes, however, require additional selection criteria, such as whether the data are to be treated as Time-Series Observations (TSO). Some EXP_TYPEs are exclusively TSO, while others depend on the value of the TSOVISIT keyword. The following table lists the pipeline modules that should get applied to various observing modes, based on these selectors. Exposure types that do not allow TSO mode are marked as “N/A” in the TSOVISIT column.

EXP_TYPE

TSOVISIT

Stage 1 Pipeline

Stage 2 Pipeline

Stage 3 Pipeline

FGS_DARK

N/A

calwebb_dark

N/A

N/A

FGS_SKYFLAT
FGS_INTFLAT

N/A

calwebb_detector1

N/A

N/A

FGS_FOCUS

N/A

calwebb_detector1

calwebb_image2

N/A

FGS_IMAGE

N/A

calwebb_detector1

calwebb_image2

calwebb_image3

FGS_ID-STACK
FGS_ID-IMAGE
FGS_ACQ1
FGS_ACQ2
FGS_TRACK
FGS_FINEGUIDE

N/A

calwebb_guider

N/A

N/A

MIR_DARKIMG
MIR_DARKMRS

N/A

calwebb_dark

N/A

N/A

MIR_FLATIMAGE
MIR_FLATIMAGE-EXT
MIR_FLATMRS
MIR_FLATMRS-EXT

N/A

calwebb_detector1

N/A

N/A

MIR_TACQ

N/A

calwebb_detector1

calwebb_image2

N/A

MIR_CORONCAL

N/A

calwebb_detector1

calwebb_image2

N/A

MIR_IMAGE

False

calwebb_detector1

calwebb_image2

calwebb_image3

True

calwebb_tso1

calwebb_tso-image2

calwebb_tso3

MIR_LRS-FIXEDSLIT

N/A

calwebb_detector1

calwebb_spec2

calwebb_spec3

MIR_LRS-SLITLESS

True

calwebb_tso1

calwebb_tso-spec2

calwebb_tso3

False

calwebb_detector1

calwebb_spec2

calwebb_spec3

MIR_MRS

N/A

calwebb_detector1

calwebb_spec2

calwebb_spec3

MIR_LYOT
MIR_4QPM

N/A

calwebb_detector1

calwebb_image2

calwebb_coron3

NRC_DARK

N/A

calwebb_dark

N/A

N/A

NRC_FLAT
NRC_LED
NRC_GRISM

N/A

calwebb_detector1

N/A

N/A

NRC_TACQ
NRC_TACONFIRM
NRC_FOCUS

N/A

calwebb_detector1

calwebb_image2

N/A

NRC_IMAGE

N/A

calwebb_detector1

calwebb_image2

calwebb_image3

NRC_CORON

N/A

calwebb_detector1

calwebb_image2

calwebb_coron3

NRC_WFSS

N/A

calwebb_detector1

calwebb_spec2

calwebb_spec3

NRC_TSIMAGE

True

calwebb_tso1

calwebb_tso-image2

calwebb_tso3

NRC_TSGRISM

True

calwebb_tso1

calwebb_tso-spec2

calwebb_tso3

NIS_DARK

N/A

calwebb_dark

N/A

N/A

NIS_LAMP
NIS_EXTCAL

N/A

calwebb_detector1

N/A

N/A

NIS_TACQ
NIS_TACONFIRM
NIS_FOCUS

N/A

calwebb_detector1

calwebb_image2

N/A

NIS_IMAGE

N/A

calwebb_detector1

calwebb_image2

calwebb_image3

NIS_AMI

N/A

calwebb_detector1

calwebb_image2

calwebb_ami3

NIS_WFSS

N/A

calwebb_detector1

calwebb_spec2

calwebb_spec3

NIS_SOSS

True

calwebb_tso1

calwebb_tso-spec2

calwebb_tso3

False

calwebb_detector1

calwebb_spec2

calwebb_spec3

NRS_DARK

N/A

calwebb_dark

N/A

N/A

NRS_AUTOWAVE
NRS_AUTOFLAT

N/A

calwebb_detector1

N/A

N/A

NRS_IMAGE
NRS_WATA
NRS_MSATA
NRS_TACONFIRM
NRS_CONFIRM
NRS_FOCUS
NRS_MIMF

N/A

calwebb_detector1

calwebb_image2

N/A

NRS_LAMP

N/A

calwebb_detector1

calwebb_nrslamp-spec2

N/A

NRS_FIXEDSLIT
NRS_IFU
NRS_MSASPEC

N/A

calwebb_detector1

calwebb_spec2

calwebb_spec3

NRS_BRIGHTOBJ

True

calwebb_tso1

calwebb_tso-spec2

calwebb_tso3