Distinguish between observations which are background subtracted and others

Question

Distinguish between observations which are background subtracted and others

renecotyfanboy opened this issue a year ago · 8 comments

At this time, when you provide observation + background to jaxspec, it assumes that the given spectrum is background subtracted. I have no idea about where I can get this information from the provided files, since I didn't find any keywords specifying it in the spectra .pha headers (I hope I just missed it).

Any ideas @sguillot @CamilleDiez @ErwanQuintin @CamilleDiez ?

Answer 1 · 2023-12-07T13:56:59.000Z

Not sure I understand your question.

I thought when a BACKFILE keyword is provided, Xspec performs the background subtraction. If there is no such keyword, then no background is subtracted and you can either add a background in Xspec with a command, or you can model your background.

Does that answer your question ?

Answer 2 · 2023-12-07T14:06:52.000Z

Sorry I should be clearer : from what I understood discussing with Erwan, when you extract a spectrum + a background spectrum with SAS, the source spectrum is subtracted with the background spectrum when generating the PHA. And the source.pha is directly the subtracted spectra (which can have negative counts if I understood well) and the background spectra. If this is a systematic behaviour, this is no problem, I'll assume that when a spectrum is given with a background, then the source spectrum has been subtracted. And from what you say this might be the case since Xspec seems to set up the appropriated metric when supplied with a source+bkg.

What bothers me here is that it is not written anywhere in the .pha file, beside the fact there is a background file supplied. A better question would be : are there situations where people would supply a NOT subtracted source spectrum + a bkg. Since I never reduced any X-ray spectral data by myself, I am a bit puzzled about the expected outputs and how to handle this

Answer 3 · 2023-12-08T16:00:24.000Z

Some updates :

- There is a set of keywords describing the spectrum

Following this memo and the website, the 'HDUCLAS2' optional keyword should either be 'TOTAL' (source + bkgd), 'NET' for a subtracted spectrum or 'BKG'.

- Not sure every fits conforms to this standard

@ErwanQuintin 's data source and bkg headers both indicate that they are 'TOTAL' spectra, even if the first is probably background subtracted and the second is just a background spectrum. This is really annoying since it most likely comes from further use of SAS which just modifies the .fits file in place without changing anything to the header

@ErwanQuintin will go through his dataset generation next Monday so we can see what is happening with SAS

Answer 4 · 2023-12-08T17:13:43.000Z

I have a Vela X-1 observation that is not bkg subtracted. The source was so bright that the bkg was dominated by the source itself (I must precise it was with pn in timing mode). So I skipped the bkg extraction step. I will check my codes on Monday to check the headers and what is happening.

Answer 5 · 2023-12-10T18:31:04.000Z

@ErwanQuintin 's data source and bkg headers both indicate that they are 'TOTAL' spectra, even if the first is probably background subtracted and the second is just a background spectrum.

If this is confirmed, I think this should be notified to XMMSAS people. I see it as clearly misleading (not to say completely wrong).

Answer 6 · 2023-12-11T10:30:23.000Z

Erwan gave me the files at different steps of the process highlighted in the SAS data extraction tutorial

Spectrum of source + background is flagged 'TOTAL' which is okay.
Spectrum of background is flagged 'TOTAL' too, which is not good according to the OGIP standard, but there is no way for SAS to know that we are extracting a background spectrum (no additional parameter to indicate this is the background extraction).
Rebinned spectrum is also flagged as 'TOTAL', even if the tutorial mentions “background subtracted”. The grouping is probably computed on the subtracted spectrum, and then the total spectrum is written in the file (or left as it is). We checked that the counts are left unchanged and that only the grouping is added to be sure there is no subtraction.

So for Erwan's data, we are modelling using the TOTAL spectrum and the BKGD spectrum, and this is almost okay if we check the 'HDUCLAS2' keyword, the background spectrum is simply not flagged.

Answer 7 · 2023-12-11T11:13:23.000Z

I confirm that with my Vela X-1 the raw and rebinned spectra of the source are flagged "HDUCLAS2=TOTAL". I don't have background files as I didn't need to extract a background region, and this is confirmed by the keyword "BACKFILE=NONE" in the header. So I think there is indeed a problem here.

Answer 8 · 2023-12-11T16:38:14.000Z

Okay, at this time #95 will allow the user to define by himself if the spectrum is bkgd-subtracted or not. I might add warnings at some points to check for consistency with the HDUCLAS2keyword, but I think this is an overkill since SAS doesn't use it.