Author: Jean Coupon.
Contributors: Alexie Leauthaud, Martin Kilbinger, Elinor Medezinski.
[NEW]: fits support added for input files (faster reading, names for input columns, and CFITSIO filters).
SWOT
is a software written in C to compute astrophysical two-point correlation functions and weighted histograms, along with fast sky-based resampling error and covariance matrix estimations.
Available estimators include:
- angular two-point correlation function (auto and cross), "w(theta)",
- projected two-point correlation function (auto and cross), "w(R)",
- galaxy-galaxy lensing, "Delta Sigma (R)",
- 3D two-point correlation function, "xi(r)",
- 3D two-point correlation function projected along the line of sight, "xi(rp, pi), wp(rp)",
- and weighted histogram (e.g. "VMAX" estimator of mass functions).
Each estimator is wrapped around a set of "divide and conquer" algorithms, mainly (but not limited to) data storage in binary trees, approximations at large angular scale, and parellelization.
Contributions:
- the core algorithm to compute the number of pairs from a kd-tree is based on Martin Kilbinger's Athena code: http://www.cosmostat.org/software/athena/.
- the galaxy-galaxy lensing core algorithm is based on Alexie Leauthaud's code (Leauthaud et al. 2010, ApJ, 709, 97).
- Elinor Medezinski helped improving and correcting a number of bugs in the cross-correlation module.
If you use this software, please cite Coupon et al. (2012, A&A, 542A, 5). A static link is also available at http://jeancoupon.com/swot.
SWOT
requires OPEN MPI (for paralellisation), GSL (for integration and random numbers), and CFITSIO libraires.
To install OPEN MPI, please visit http://www.open-mpi.org/. Note that OPEN MPI is a wrapper to the default C-code compiler on your machine. To install it with another C compiler (for example intel icc
), install OPEN MPI with the following command:
$ ./configure --prefix=YOUR/PATH CC=icc CXX=icpc
To download and install GSL, please visit http://www.gnu.org/software/gsl/.
To download and install CFITSIO, please visit http://heasarc.gsfc.nasa.gov/docs/software/fitsio/fitsio.html.
- Download the latest version here https://github.com/jcoupon/swot/releases/latest
- Untar the archive and change directory to
swot-X.Y.Z
- Run
Make
- A test suite is available here.
If OPEN MPI or GSL are installed in a different directory than /usr/local
, edit the Makefile
and set the MPI
or GSL
variable accordingly, or run:
$ make GSL=path/to/gsl MPI=path/to/mpi
The binary file is installed in swot-X.Y.Z/bin. Simply update your PATH
variable or move swot
to your working directory.
venice
is a companion software to draw random points out of DS9 or fits masks. See https://github.com/jcoupon/venice.
Run the software:
$ mpirun -np [Ncpu] swot -c configFile -corr ESTIMATOR [options]:
Display the default configuration file:
$ swot -d: display a default configuration file
Important: if using "RADEC" coordinates, the angle in the input catalogues must be in decimal degrees.
auto
,cross
: two-point correlation function. The choice for the estimator (-est
) can be made among:ls
: Landy & Szalay (1993) estimator (default),nat
: natural estimator: 1+DD/RR,ham
: Hamilton (1993), orpeebles
: Peebles (1993).gglens
auto_wp
,cross_wp
auto_3D
,cross_3D
number
From Athena's documentation:
"If two nodes see each other under angles which are smaller than the open-angle threshold (OATH in config file), the tree is not further descended and those nodes are correlated directly. The mean, weighted galaxy ellipticities of the both nodes are multiplied, the angular separation is the binned barycenter distance. This is of course very time-saving, which is the great advantage of a tree code over a brute-force approach. It introduces however errors in the correlation function, in particular on large scales, because the galaxy ellipticities are smeared out. One should play around with the value of OATH to see its influence on the results. As a guideline, OATH = 0.05 (about 3 degrees) can be used."
-weighted [yes,no]
: compute weighted quantities ([X]: column for the weights if available). WARNING: if set, all input files must contain a weight column (including the random samples, put 1.0's if no weight).
The input format is set with the cols[1,2]
and rancols[1,2]
options (the first column is "1"):
-
default input format for
number
RA DEC m [weight]
: change column ids with:-cols1 1,2,3[,4]
, withm
the quantity to compute the histogram for. Positions on the sky are required for the computation of sky-based resampling errors. -
default input format for
auto
andcross
[-proj como/phys] [-weighted]
:RA DEC [z] [weight]
. This means ->RA DEC z
(if-proj phys
),RA DEC w
(if-weighted
), orRA DEC z w
(if-proj phys -weighted
). Change column ids with:-cols1 1,2[,3][,4]
-
default input format for
auto_3D
andcross_3D
[-weighted]
:X Y Z [weight]
. Change column ids with:-cols1 1,2,3[,4]
. Trick: for 2D cartesian correlations, simply fix the 3rd column to a constant number. -
default input format for
auto_wp
andcross_wp
:RA DEC z [weight]
. Change column ids with:-cols1 1,2,3[,4]
-
default input format for
gglens
. Lenses (-cols1
)RA DEC z sigz
. Change column ids with:-cols1 1,2,3,4
. Sources (-cols2
)RA DEC z sigz e1 e2 weight
. Change column ids with:-cols2 1,2,3,4,5,6,7
. -
default input format for
gglens
to compute calibration factor (-calib yes
). Lenses (-cols1
)RA DEC z sigz
. Change column ids with:-cols1 1,2,3,4
. Sources (-cols2
)RA DEC z sigz calib e2 weight
. Change column ids with:-cols2 1,2,3,4,5,6,7
, where calib = 1+m or c.
Same as above, except that column ids may be replaced by their actual names.
-fits
: [auto,yes,no]. If "auto", will interpret files with ".fits" or ".fit" as binary fits tables.
Example:
$ mpirun -np 8 swot -c configFile -corr auto -data1 'fileIn.fits[RA>30.0]' -cols1 RA,DEC \
-ran1 'fileRanIn.fits[RA>30.0&&#row%10==0]' -rancols1 RA,DEC
The [RA>30.0&&#row%10==0]
filter will select one in 10 random objects and with RA coordinate larger than 30.0. Note that with Mac OS, spaces seem not to be allowed.
More examples from the CFITSIO documentation:
[ binary && mag <= 5.0]
: Extract all binary stars brighter than fifth magnitude (note that the initial space is necessary to prevent it from being treated as a binning specification).
[#row >= 125 && #row <= 175]
: Extract row numbers 125 through 175.
[IMAGE[4,5] .gt. 100]
: Extract all rows that have the (4,5) component of the IMAGE column greater than 100.
[abs(sin(theta * #deg)) < 0.5]
: Extract all rows having the absolute value of the sine of theta less than a half where the angles are tabulated in degrees.
[SUM( SPEC > 3*BACKGRND )>=1]
: Extract all rows containing a spectrum, held in vector column SPEC, with at least one value 3 times greater than the background level held in a keyword, BACKGRND
[VCOL=={1,4,2}]
: Extract all rows whose vector column VCOL contains the 3-elements 1, 4, and 2.
[@rowFilter.txt]
: Extract rows using the expression contained within the text file rowFilter.txt.
[gtifilter()]
: Search the current file for a GTI extension, filter the TIME column in the current table, using START/STOP times taken from columns in the GTI extension.
[regfilter("pow.reg")]
: Extract rows which have a coordinate (as given in the X and Y columns) within the spatial region specified in the pow.reg region file.
[regfilter("pow.reg", Xs, Ys)]
: Same as above, except that the Xs and Ys columns will be used to determine the coordinate of each row in the table.
See also http://heasarc.gsfc.nasa.gov/docs/software/fitsio/filters.html.
-nbins
: controls the number of bins.
-nbins_pi
: controls the number of bins in the "pi" direction for wp(rp).
printTree
: outputs the tree structure in "out".data[1,2]_tree.[ascii,fits]. Output format in ascii or fits, depending on input file format. The recorded columns are "dim1 dim2 ... dimNDIM sub_weights1 sub_weights2 ... sub_weightsNamples sub_id rank".
printTreeAndExit
: same as above but exists after.
-silent yes
: shuts off all messages.
To reduce the memory usage one can reduce the number of resamplings samples (-nsamples 8) for example, but the covariance matrix is likely to be inaccurate.
Maximum efficiency is reached when the number of cpus is a power of two.
(Workflow taken from https://github.com/github/gitignore)
- Fork this project to your account.
- Create a branch for the change you intend to make.
- Make your changes to your fork.
- Send a pull request from your fork’s branch to our
master
branch.
Using the web-based interface to make changes is fine too, and will help you by automatically forking the project and prompting to send a pull request too.