/Rbeast

Bayesian Change-Point Detection and Time Series Decomposition

Primary LanguageC

BEAST: A Bayesian Ensemble Algorithm for Change-Point Detection and Time Series Decomposition

BEAST (Bayesian Estimator of Abrupt change, Seasonality, and Trend) is a fast, generic Bayesian model averaging algorithm to decompose time series or 1D sequential data into individual components, such as abrupt changes, trends, and periodic/seasonal variations, as described in Zhao et al. (2019). BEAST is useful for changepoint detection (e.g., breakpoints, structural breaks, regime shifts, or anomalies), trend analysis, time series decomposition (e.g., trend vs seasonality), time series segmentation, and interrupted time series analysis. See a list of selected studies using BEAST .

Quick Installation

BEAST was impemented in C/C++ but accessible from R, Python, Matlab, and Octave. Run the following to install:

  • Python: pip install Rbeast
  • Matlab: eval(webread('http://b.link/rbeast',weboptions('cert','')))
  • Octave: eval(webread('http://b.link/rbeast'))
  • R lang: install.packages("Rbeast")

Quick Usage

One-liner code for Python, Matlab and R:

# Python example
import Rbeast as rb; (Nile, Year)=rb.load_example('nile'); o=rb.beast(Nile,season='none'); rb.plot(o)

# Matlab/Octave example
load('Nile'); o = beast(Nile, 'season','none'); plotbeast(o)

# R example
library(Rbeast); data(Nile); o = beast(Nile); plot(o)

Installation for R

Rbeast in CRAN-TASK-VIEW: [Time Series Analysis] [Bayesian inference] [Environmetrics]

An R package Rbeast has been deposited at CRAN. ( On CRAN, there is another Bayesian time-series package named "beast", which has nothing to do with the BEAST algorithim. Our package is Rbeast. Also, Rbeast has nothing to do with the famous "Bayesian evolutionary analysis by sampling trees" aglorithm.) Install Rbeast in R using

install.packages("Rbeast")

Run and test Rbeast in R

The main functions in Rbeast are beast(Y, ...), beast.irreg(Y, ...), and beast123(Y, metadata, prior, mcmc, extra). The code snippet below provides a starting point for the basic usage.

library(Rbeast)
data(Nile)                       #  annual streamflow of the Nile River    
out = beast(Nile, season='none') #  'none': trend-only data without seasonlaity   
print(out)                   
plot(out)
?Rbeast                          # See more details about the usage of `beast`    
     
tetris()                         # if you dare to waste a few moments of your life 
minesweeper()                    # if you dare to waste a few more moments of your life 

Installation for Matlab

View Rbeast on File Exchange

Install the Matlab version of BEAST automatically to a local folder of your choice by running

beastPath = 'C:\beast\'                   
eval( webread('http://b.link/rbeast') )  

%%%%%%%%%%%%%%%%%%%%%%%%%%% Note on Automatic Installtion %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% 
% 1. Write permission needed for your chosen path; the variable name must be 'beastPath'.   %
% 2. If webread has a certificate error, run the following line instead:                    %
    eval(  webread( 'http://b.link/rbeast', weboptions('cert','') )  )                       %
% 3. If the automatic installation fails, please manually download all the files (see blow) %       
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

The above will download all the files in the Rbeast\Matlab folder at Github to the chosen folder: if beastPath is missing, a default temporary folder (e.g., C:\Users\$user_name$\AppData\Local\Temp\Rbeast for Windows 10) will be used. If the automatic script fails, please download the Matlab files from Github manually. These files include a Matlab mex library compiled from the C soure code (e.g., Rbeast.mexw64 for Windows, Rbeast.mexa64 for Linux, Rbeast.mexmaci64 for MacOS) and some Matlab wrapper functions (e.g.,beast.m, and beast123.m) similar to the R interface, as well as some test datasets (e.g., Nile.mat, and co2.mat).

We generated the Matlab mex binary library on our own machines with Win10, Ubuntu 22.04, and macOS High Sierra. If they fail on your machine, the mex library can be compiled from the C source code files under Rbeast\Source. If needed, we are happy to work with you to compile for your specific OS or machines. Additional information on compilations from the C source is also given below.

Run and test Rbeast in Matlab

The Matlab API is similar to those of R. Below is a quick example:

 help beast
 help beast123  
 load('Nile.mat')                                   % annual streamflow of the Nile River startin from year 1871
 out = beast(Nile, 'season', 'none','start', 1871)  % trend-only data without seasonality
 printbeast(out)
 plotbeast(out)

Installation for Octave

The same as for Matlab. Now, only Windows platforms are supported. If needed for other platforms (e.g., Octave in Linux and Mac), please contact the author at zhao.1423@osu.edu for support.


Installation for Python

A package Rbeast has been deposited at PyPI: https://pypi.org/project/Rbeast/. Run the command below in a console to install:

  pip install Rbeast

Binary wheel files were built on Windows, MacOS, and Linux for Python version 3.7 to 3.11 (either x86_64 or arm64 CPU). If the installation fails, please install from the source to build the package using pip install Rbeast --no-binary :none:, which requies a C/C++ compliler (e.g., requiring gcc on Linux or xcode on Mac). If needed, contact Kaiguang Zhao (zhao.1423@osu.edu) to help build the package for your OS platform and Python version.

Run and test Rbeast in Python

Nile is annual streamflow of the River Nile, starting from Year 1871. As annual observations, it has no periodic component (i.e., season='none').

import Rbeast as rb                                       # Import the Rbeast package as `rb`
nile, year = rb.load_example('nile')                      # a sample time series
o          = rb.beast( nile, start=1871, season='none')
rb.plot(o)
rb.print(o)
o  # see a list of output fields in the output variable o

The second example googletrend is a monthly time series of the Google Search popularity of the word beach over the US. This monthly time series is reguarly-spaced (i.e., deltat=1 month =1/12 year); it has a cyclyic component with a period of 1 year. That is, the number of data points per period is period / deltat = 1 year / 1 month = 1/(1/12) = 12.

beach, year = rb.load_example('googletrend')
o = rb.beast(beach, start= 2004.0, deltat=1/12, period = 1.0)       # the time unit is unknown or arbitrary
o = rb.beast(beach, start= 2004.0, deltat=1/12, period ='1.0 year') # the time unit is fractional year
o = rb.beast(beach, start= 2004.0, deltat='1 month', period =1.0)   # the time unit is fractional year
rb.plot(o)
rb.print(o)

Installation for Julia/IDL (yet to come)

Wrappers in Julia and IDL are being developed: We welcome contributions and help from interested developers. If interested, contact Kaiguang Zhao at zhao.1423@osu.edu.

Description of BEAST

Interpretation of time series data is affected by model choices. Different models can give different or even contradicting estimates of patterns, trends, and mechanisms for the same data–a limitation alleviated by the Bayesian estimator of abrupt change,seasonality, and trend (BEAST) of this package. BEAST seeks to improve time series decomposition by forgoing the "single-best-model" concept and embracing all competing models into the inference via a Bayesian model averaging scheme. It is a flexible tool to uncover abrupt changes (i.e., change-points), cyclic variations (e.g., seasonality), and nonlinear trends in time-series observations. BEAST not just tells when changes occur but also quantifies how likely the detected changes are true. It detects not just piecewise linear trends but also arbitrary nonlinear trends. BEAST is applicable to real-valued time series data of all kinds, be it for remote sensing, finance, public health, economics, climate sciences, ecology, and hydrology. Example applications include its use to identify regime shifts in ecological data, map forest disturbance and land degradation from satellite imagery, detect market trends in economic data, pinpoint anomaly and extreme events in climate data, and unravel system dynamics in biological data. Details on BEAST are reported in Zhao et al. (2019). The paper is available at https://go.osu.edu/beast2019.

Note on computation

As a Bayesian algorithm, BEAST is fast and is possibly among the fastest implementations of Bayesian time-series analysis algorithms of the same nature. (But it is still slower, compared to nonBayesian methods.) For applications dealing with a few to thousands of time series, the computation won't be an practical concern. But for remote sensing/geospatial applications that may easily involve millions or billions of time series, computation will be a big challenge for Desktop computer users. We suggest first testing BEAST on a single time series or small image chips first to determine whether BEAST is appropriate for your applications and, if yes, estimate how long it may take to process the whole image.

In any case, for those users handling stacked time-series images, do not use beast or beast.irreg. Use beast123 instead, which can handle 3D data cubes and allow parallel computing. We also welcome consultation with Kaiguang Zhao (zhao.1423@osu.edu) to give specific suggestions if you see some value of BEAST for your applications.

Reference

Compilation from C source code (for developers and experts only)

Though not needed but if preferred, the code can be compiled for your specific machines. Check the Rbeast\Source folder at GitHub for details.

Reporting Bugs or getting help

BEAST is distributed as is and without warranty of suitability for application. The one distributed above is still a beta version, with potential room for further improvement. If you encounter flaws with the software (i.e. bugs) please report the issue. Providing a detailed description of the conditions under which the bug occurred will help to identify the bug, you can directly email its maintainer Dr. Kaiguang Zhao at zhao.1423@osu.edu. Alternatively, Use the Issues tracker on GitHub to report issues with the software and to request feature enhancements.

Acknowledgement:

BEAST is developed by Yang Li, Tongxi Hu, Xuesong Zhang, and Kaiguang Zhao. The development of BEAST received supported through Microsoft Azure for Research (CRM0518513) and a USGS 104B grant and a Harmful Algal Bloom Research Initiative grant from the Ohio Department of Higher Education. The contribution of Xuesong Zhang was supported through USDA-ARS.

Selected publications using BEAST/Rbeast

Despite being published originally for ecological and enviornmental applications, BEAST is developed as a generic tool applicable to time series or time-series-like data arising from all disciplines. BEAST is not a heuristic algorithm but a rigorous statistical model. Below is a list of selected peer-reviewed pulications that used BEAST for statistical data analysis.

Discipline Publication Title
Remote Sensing Li, J., Li, Z., Wu, H., and You, N., 2022. Trend, seasonality, and abrupt change detection method for land surface temperature time-series analysis: Evaluation and improvement. Remote Sensing of Environment, 10.1016/j.rse.2022.113222
Paleoclimatology Anastasia Zhuravleva et al., 2023. Caribbean salinity anomalies contributed to variable North Atlantic circulation and climate during the Common Era. Science Advances, DOI:10.1126/sciadv.adg2639
Population Ecology Henderson, P. A. (2021). Southwood's Ecological Methods (5th edition). Oxford University Press., page 475-476
Cardiology Ozier, D., Rafiq, T., de Souza, R. and Singh, S.M., 2023. Use of Sacubitril/Valsartan Prior to Primary Prevention Implantable Cardioverter Defibrillator Implantation. CJC Open.
Anthropocene Science Thomas, E.R., Vladimirova, D.O., Tetzner, D.R., Emanuelsson, D.B., Humby, J., Turner, S.D., Rose, N.L., Roberts, S.L., Gaca, P. and Cundy, A.B., 2023. The Palmer ice core as a candidate Global boundary Stratotype Section and Point for the Anthropocene series. The Anthropocene Review, p.20530196231155191.
Political Science Reuning, K., Whitesell, A. and Hannah, A.L., 2022. Facebook algorithm changes may have amplified local republican parties. Research & Politics, 9(2), p.20531680221103809.
Watershed Hydrology Sakizadeh, M., Milewski, A. and Sattari, M.T., 2023. Analysis of Long-Term Trend of Stream Flow and Interaction Effect of Land Use and Land Cover on Water Yield by SWAT Model and Statistical Learning in Part of Urmia Lake Basin, Northwest of Iran. Water, 15(4), p.690.
Hydraulic Engineering Xu, X., Yang, J., Ma, C., Qu, X., Chen, J. and Cheng, L., 2022. Segmented modeling method of dam displacement based on BEAST time series decomposition. Measurement, 202, p.111811.
Social Media Barrie, C., Ketchley, N., Siegel, A. and Bagdouri, M., 2023. Measuring Media Freedom.
Political Economy Benchimol, J. and Palumbo, L., 2023. Sanctions and Russian Online Prices.
Physiology Shakeel, M., Brockmann, A. Temporal effects of sugar intake on fly local search and honey bee dance behaviour. J Comp Physiol A (2023). https://doi.org/10.1007/s00359-023-01670-6
Ichthyology Kaeding, L.R., 2023. Climate-change and nonnative-piscivore impacts on a renowned Oncorhynchus metapopulation, requirements for metapopulation recovery, and similarities to anadromous salmonid metapopulations. Aquatic Sciences, 85(4), p.88.
Remote Sensing Mulverhill, C., Coops, N.C. and Achim, A., 2023. Continuous monitoring and sub-annual change detection in high-latitude forests using Harmonized Landsat Sentinel-2 data. ISPRS Journal of Photogrammetry and Remote Sensing, 197, pp.309-319.
Physical Chemistry Faran, M. and Bisker, G., 2023. Nonequilibrium Self-Assembly Time Forecasting by the Stochastic Landscape Method. The Journal of Physical Chemistry B.
Analytical Chemistry Šimić, M., Neuper, C., Hohenester, U. and Hill, C., 2023. Optofluidic force induction as a process analytical technology. Analytical and Bioanalytical Chemistry, pp.1-11.
Ecosystem Sciences Lyu, R., Zhao, W., Pang, J., Tian, X., Zhang, J. and Wang, N., 2022. Towards a sustainable nature reserve management: Using Bayesian network to quantify the threat of disturbance to ecosystem services. Ecosystem Services, 58, p.101483.
Environmental Sciences Nickerson, S., Chen, G., Fearnside, P., Allan, C.J., Hu, T., de Carvalho, L.M. and Zhao, K., 2022. Forest loss is significantly higher near clustered small dams than single large dams per megawatt of hydroelectricity installed in the Brazilian Amazon. Environmental Research Letters.
Geology Fan, X., Goeppert, N. and Goldscheider, N., 2023. Quantifying the historic and future response of karst spring discharge to climate variability and change at a snow-influenced temperate catchment in central Europe. Hydrogeology Journal, pp.1-17.
Wildlife Smith, Matthew M., and Jonathan N. Pauli. "Connectivity maintains genetic diversity and population persistence within an archipelagic refugia even under declining lake ice." Mechanisms of species recovery for a forest carnivore in a changing landscape: 173.
Climate Sciences Duke, N.C., Mackenzie, J.R., Canning, A.D., Hutley, L.B., Bourke, A.J., Kovacs, J.M., Cormier, R., Staben, G., Lymburner, L. and Ai, E., 2022. ENSO-driven extreme oscillations in mean sea level destabilise critical shoreline mangroves—An emerging threat. PLOS Climate, 1(8), p.e000003
Finance Candelaria, Christopher A., Shelby M. McNeill, and Kenneth A. Shores. (2022). What is a School Finance Reform? Uncovering the ubiquity and diversity of school finance reforms using a Bayesian changepoint estimator.(EdWorkingPaper: 22-587). Retrieved from Annenberg Institute at Brown University: https://doi.org/10.26300/4vey-3w10
Public health Linnell, K., Fudolig, M., Schwartz, A., Ricketts, T.H., O'Neil-Dunne, J.P., Dodds, P.S. and Danforth, C.M., 2022. Spatial changes in park visitation at the onset of the pandemic. arXiv preprint arXiv:2205.15937.
Biometerology Li, Y., Liu, Y., Bohrer, G., Cai, Y., Wilson, A., Hu, T., Wang, Z. and Zhao, K., 2022. Impacts of forest loss on local climate across the conterminous United States: Evidence from satellite time-series observations. Science of The Total Environment, 802, p.149651.
Applied Math Ferguson, Daniel, and François G. Meyer. Probability density estimation for sets of large graphs with respect to spectral information using stochastic block models. arXiv preprint arXiv:2207.02168 (2022).
Transportation Science Delavary, M., Kalantari, A.H., Mohammadzadeh Moghaddam, A., Fakoor, V., Lavallière, M. and Wilhelm Siebert, F., 2023. Road traffic mortality in Iran: longitudinal trend and seasonal analysis, March 2011-February 2020. International Journal of Injury Control and Safety Promotion, pp.1-12.
Water quality He, Ziming, Jiayu Yao, Yancen Lu, and Danlu Guo. "Detecting and explaining long‐term changes in river water quality in south‐eastern Australia." Hydrological Processes: e14741.
Air quality Wu, S., Yao, J., Wang, Y. and Zhao, W., 2023. Influencing factors of PM2. 5 concentrations in the typical urban agglomerations in China based on wavelet perspective. Environmental Research, p.116641.
Hydrology Zohaib, M. and Choi, M., 2020. Satellite-based global-scale irrigation water use and its contemporary trends. Science of The Total Environment, 714, p.136719.
Energy Engineering Lindig, S., Theristis, M. and Moser, D., 2022. Best practices for photovoltaic performance loss rate calculations. Progress in Energy, 4(2), p.022003.
Virology Shen, L., Sun, M., Song, S., Hu, Q., Wang, N., Ou, G., Guo, Z., Du, J., Shao, Z., Bai, Y. and Liu, K., 2022. The impact of anti-COVID19 nonpharmaceutical interventions on hand, foot, and mouth disease—A spatiotemporal perspective in Xi'an, northwestern China. Journal of medical virology.
Pharmaceutical Sciences Patzkowski, M.S., Costantino, R.C., Kane, T.M., Nghiem, V.T., Kroma, R.B. and Highland, K.B., 2022. Military Health System Opioid, Tramadol, and Gabapentinoid Prescription Volumes Before and After a Defense Health Agency Policy Release. Clinical Drug Investigation, pp.1-8.
Geography Cai, Y., Liu, S. and Lin, H., 2020. Monitoring the vegetation dynamics in the Dongting Lake Wetland from 2000 to 2019 using the BEAST algorithm based on dense Landsat time series. Applied Sciences, 10(12), p.4209.
Oceanography Pitarch, J., Bellacicco, M., Marullo, S. and Van Der Woerd, H.J., 2021. Global maps of Forel–Ule index, hue angle and Secchi disk depth derived from 21 years of monthly ESA Ocean Colour Climate Change Initiative data. Earth System Science Data, 13(2), pp.481-490.
Photovoltaics Micheli, L., Theristis, M., Livera, A., Stein, J.S., Georghiou, G.E., Muller, M., Almonacid, F. and Fernández, E.F., 2021. Improved PV soiling extraction through the detection of cleanings and change points. IEEE Journal of Photovoltaics, 11(2), pp.519-526.
Climate Sciences White, J.H., Walsh, J.E. and Thoman Jr, R.L., 2021. Using Bayesian statistics to detect trends in Alaskan precipitation. International Journal of Climatology, 41(3), pp.2045-2059.
Field Hydrology Merk, M., Goeppert, N. and Goldscheider, N., 2021. Deep desiccation of soils observed by long-term high-resolution measurements on a large inclined lysimeter. Hydrology and Earth System Sciences, 25(6), pp.3519-3538.
Forest Ecology Moreno-Fernández, D., Viana-Soto, A., Camarero, J.J., Zavala, M.A., Tijerín, J. and García, M., 2021. Using spectral indices as early warning signals of forest dieback: The case of drought-prone Pinus pinaster forests. Science of The Total Environment, 793, p.148578.
Atmospheric Sciences Tingwei, C., Tingxuan, H., Bing, M., Fei, G., Yanfang, X., Rongjie, L., Yi, M. and Jie, Z., 2021. Spatiotemporal pattern of aerosol types over the Bohai and Yellow Seas observed by CALIOP. Infrared and Laser Engineering, 50(6), p.20211030.
Terrestrial ecology Dashti, H., Pandit, K., Glenn, N.F., Shinneman, D.J., Flerchinger, G.N., Hudak, A.T., de Graaf, M.A., Flores, A., Ustin, S., Ilangakoon, N. and Fellows, A.W., 2021. Performance of the ecosystem demography model (EDv2. 2) in simulating gross primary production capacity and activity in a dryland study area. Agricultural and Forest Meteorology, 297, p.108270.
Statistics Storath, M. and Weinmann, A., 2023. Smoothing splines for discontinuous signals. Journal of Computational and Graphical Statistics, (just-accepted), pp.1-26.
Environmental Engineering Bainbridge, R., Lim, M., Dunning, S., Winter, M.G., Diaz-Moreno, A., Martin, J., Torun, H., Sparkes, B., Khan, M.W. and Jin, N., 2022. Detection and forecasting of shallow landslides: lessons from a natural laboratory. Geomatics, Natural Hazards and Risk, 13(1), pp.686-704.
Hydrology Yang, X., Tian, S., You, W. and Jiang, Z., 2021. Reconstruction of continuous GRACE/GRACE-FO terrestrial water storage anomalies based on time series decomposition. Journal of Hydrology, 603, p.127018.
Landscape Ecology Adams, B.T., Matthews, S.N., Iverson, L.R., Prasad, A.M., Peters, M.P. and Zhao, K., 2021. Spring phenological variability promoted by topography and vegetation assembly processes in a temperate forest landscape. Agricultural and Forest Meteorology, 308, p.108578.