A Tool for Time Series Reconstruction of Large-Scale Astronomical Catalogues
This is the main source code of AstroCatR, which can reconstruct all celestial objects' time series data for astronomical catalogues.
AstroCatR is a command-line opensource program running on the Linux platform, which is implemented in C and Python. Its capabilities are based on specialized sky partitioning and MPI parallel programming.
AstroCatR contains three parts, ETL (extract-transform-load) preprocessing, TS-matching calculation and time series data retrieval. You need to have original catalogs, and small data samples are provided in the Data directory. Then run program ETL preprocessing to generate sky zoning file. Next, run program TS-matching to mark celestial objects. Finally, run program Query to search the celestial objects from time series datasets which matched with the target.
CMP directory is the comparative experiments for in-memory reference table, which uses MySQL's memory table and PostgreSQL's unlogged table to store the reference table for matching calculation.
This program has the following dependencies, which can be found in main directory.
Cfitsio is used to parse catalogue FITS files information, MPI is used to accelerate TS-matching calculations and Python is used to manage user queries. Gnuplot creates scatter plots according to the time series data.
Installation steps of Cfitsio
tar zxvf cfitsio_latest.tar.gz
Generate the make file:
Note that I have included the option --prefix=/usr in order to control where the library will be installed. Otherwise, by default CFITSIO will be installed under the source directory, which normally is a bad idea because that directory will not be included in the default search path for auto-tools.
Compile the source files:
make make install
The different versions of the CFITSIO library (libcfitsio.*) are installed under: /usr/lib. The auxiliary files longnam.h, fitsio.h, fitsio2.h, and drvrsmem.h are placed under: /usr/include.
1. Folder: ETL Preprocessing
Under the xingbiao folder, use the next command to compile
It will generate an executable file：xingbiao1, then the ETL preprocessing can be executed by running a shell script.
2. Folder: Matching Calculation
Use the next command to compile
It will generate an executable file：Matching
It is written in Python, so it needs to install Python and the Python version must be greater than 2.7.
1. The ETL Preprocessing
Set up your own configuration file and create a new folder.
Setting Paths according to actual requirements and execute corresponding shell scripts.
time sh test.sh
To generate sky zoning files
2. TS-matching Calculation
Set the number of processes and level of partitions according to the actual situation, and achieve the desired performance based on partition function.
Use the next command to run mactching operation
mpirun -np $procs ./Matching $input $output $procs
Set position parameters and use next command to query service
python Query.py $ra $dec $input $output
Use next command to produce scatter plots
time sh Draw.sh
Use the following command to realize the transfer of time series data
python TScat.py $input $output