Multiwavelength cross-match and maximum likelihood for the LOFAR Surveys Tier 1.
The software requires Python 3.4 or higher to run the ML estimation in parallel. Please notice that running the code in big datasets requires a considerable amount of memory.
The dependencies are:
- astropy
- numpy
- wquantiles
- pandas (used to estimate the i-band magnitude)
- tqdm
- scipy (optional; used in some likelihood ratio threshold methods)
We selected the AllWISE Source Catalog catalogue
All sky query is selected and the constrains in ra and dec are included.
We use the DR1 of PanSTARRS and download the data using the CasJobs interface.
The query used for the Tier 1 region is in: query
The data PanSTARRS and WISE are matched using a Maximum Likelihood (ML) method.
The steps are the following:
- Filtering of the catalogues: notebook
- Correct the reddening: notebook
- Computing of Q0: notebook
- Estimation of the ML parameters: notebook
- Application of the ML matching: notebook
A final catalogue with all the sources (matched and non-matched) called pw.fits
is produced at the end.
A Maximum Likelihood method is also applied to LOFAR sources and sources in the combined WISE-PanSTARRS catalogue.
Before applying the ML matching we corrected an error in the format of the pw catalogue with this notebook
The
The output of the notebook lofar_params.pckl is used as the initial input for the notebook Match_LOFAR_combined-iteration.ipynb. In this last notebook the ML paremeters are adjusted iteratively until they converge (currently in 5 iterations). The output of this notebook is located in the directory idata/main. The lofar_params_.pckl (being "n" the number of the last iteration and the bigger number found in the directory) should be manually moved to the main directory and renamed to "lofar_params.pckl". This will be the parameters used for the ML matching notebooks.
We discarded the extended sources with a major axis bigger than 30 arcseconds to compute the parameters.
The ML matching is shown in: notebook.
The matching can be applied to any input catalogue. We applied it to the Gaussians catalogue in: notebook. This notebook can be used to match to other catalogues as well.
There is also a version that saves all the matches avobe the selected ML threshold. In this case, if a source is matched by two or more WISE-PanSTARRS sources with a ML above the threshold, all the matches are saved: notebook.
There are a couple of obsolete notebooks that are no longer used:
The main auxiliary code used by the ML estimators is in mltier1.py
It is possible to execute automatically the notebooks without the need of a browser. This is very useful for the automated execution of pipelines or for the running of heavy computations (some of the notebooks take several hours or even days to run). To run automatically a notebook you can use:
jupyter nbconvert --to notebook \\
--ExecutePreprocessor.timeout=6000 \\
--kernel_name python3 \\
--execute PanSTARRS_WISE_reddening.ipynb
adapting the kernel name and timeout.