Subsections

• 5.13.1 Overlap Rejection
• 5.13.2 Optimization for Source Validity (PROB_NO_SOURCE statistic)
• 5.13.3 Optimization for Source Position
• 5.13.4 Optimization for Photometry and Spectral Fitting

5.13 Discarding Observations

In a multi-observation analysis, single-observation extractions are combined to form multi-ObsId data products and source properties, as described in the previous sections. However, once the alternate strategy of merging extractions is adopted it would be foolish to blindly merge all the extractions available for every source.

Consider, for example, a source that has two extractions--one beautiful observation on-axis using a small aperture containing almost zero background, and a second observation far off-axis using a huge aperture containing many background counts. If the source is very bright and the off-axis observation is not crowded, then the source properties are most accurately estimated by merging the two extractions, since the extra signal gained from the off-axis observation is more important than the relatively insignificant additional background that is suffered. However, if the source is very weak, then retaining the horrible off-axis observation would utterly corrupt the quality of the source properties--the merged extractions may even produce negative net counts (by chance). Clearly, an observer tackling misaligned Chandra pointings must make some sort of decision about which source extractions should be discarded. AE implements several algorithms for discarding extractions, described in the following subsections.

The notion of discarding data may seem strange at first, but observers routinely discard ACIS data that was obtained during periods of very high instrumental background (due to solar activity). For convenience, this is commonly done in the early stages of data analysis, guided by the damage the enhanced background will do to the most sensitive sources (e.g., diffuse sources). A somewhat better strategy would be to choose high-background periods to discard on a source-by-source basis, since very bright sources would benefit more from extra integration time than from a reduction to their already insignificant background. AE has not yet adopted this optimum strategy for time filtering because implementation is difficult.

5.13.1 Overlap Rejection

All merges will discard extractions that suffer extreme crowding because in those conditions we do not have confidence that we can effectively estimate backgrounds for each extraction. Crowding is measured by an OVERLAP metric assigned to each extraction by the ae_make_catalog tool (§7.5), quantifying how much the extraction region overlaps its nearest neighbor (closely-spaced sources can still overlap substantially even though AE automatically reduces the extraction region to minimize overlap; this is because we have imposed a floor in that reduction so that we never have an extraction region containing <40% of the full PSF). The MERGE stage (§7.8) accepts an OVERLAP_LIMIT theshold parameter; extractions exceeding this threshold are considered to have excessive overlap and will be discarded.

5.13.2 Optimization for Source Validity (PROB_NO_SOURCE statistic)

When the observer is interested in the validity of proposed sources using AE's PROB_NO_SOURCE statistic (the probability that the source is just a background fluctuation; §5.10.3), then we recommend an AE option that selects whatever subset of the available extractions optimizes (minimizes) PROB_NO_SOURCE for each source. A slogan that describes this strategy would be ``I believe a source exists if it is significant in any observation, or in any combination of observations''. For highly variable astrophysical sources, such as young stars, this is a reasonable strategy even when the pointings are aligned (since the non-flaring observations can be ignored). An extraction much further off-axis than its peers will be included only when it contributes ``more'' signal than background.

AE's algorithm is simply an extension of the common practice of searching for sources within each ObsID separately, and also within observer-defined combinations of ObsIDs. All such approaches increase a project's sensitivity (especially to variable sources) at the expense of an increased false detection rate (from the additional number of random ``trials'' that can produce spurious detections).

This algorithm is chosen by the /MERGE_FOR_PB option to the MERGE stage (§7.8).

5.13.3 Optimization for Source Position

When the observer is interested in the position of sources, then we recommend an AE option that selects whatever subset of the available extractions optimizes (minimizes) the expected position uncertainty (§5.3) for each source. A slogan that describes this strategy would be ``I believe all observations of a source are well aligned, so let's estimate the position using only the best data we have''. An extraction much further off-axis than its peers will be included only when the increase in the number of merged counts outweighs the increase in the spread of the merged counts.

This algorithm is chosen by the /MERGE_FOR_POSITIONS option to the MERGE stage (§7.8).

5.13.4 Optimization for Photometry and Spectral Fitting

When the observer is interested in time-averaged photometric properties (e.g., fluxes, spectra) then selecting the extractions to merge becomes more dangerous. Any time extractions are discarded in order to optimize a photometric quantity (e.g., signal-to-noise ratio) a bias can be introduced into all photometric properties because the discarded extraction may have, merely by chance, fewer observed counts from the source than the long-term average. Thus, the observer must balance two undesirable outcomes: sources whose photometry is spoiled by including very poor-quality extractions, and sources whose photometry suffers the suspicion of bias because some extractions were discarded after looking at their data.

AE offers an option that strikes this balance by discarding extractions only when retaining them would drive the signal-to-noise ratio of the merged data set significantly below the optimal signal-to-noise ratio. The observer specifies the minimum acceptable ratio between the signal-to-noise ratio achieved by the merge and the optimal signal-to-noise ratio achievable by discarding more extractions. A slogan that describes this strategy would be ``I will tolerate some, but not too much, damage to SNR in order to avoid the specter of bias in the photometry''.

This algorithm is chosen by the /MERGE_FOR_PHOTOMETRY option to the MERGE stage (§7.8).