Time Series - Science topic

Sorry Benazir Mahar, I did the wrong answer to you, i was confused with another question, Thanks for your answer

Dear Wudu Berie

I consider the implications of your proposed research topic to be of much importance to the economic development of your country and the academic community as well. However, I think it would be more appropriate if you can be more specific on the kind of help you need?

Do you need an expert to help you structure your introduction or share brains with you in achieving the overall objective of your study?

May be EWMA

There are no movements or they are cyclical with a very high oscillation frequency.

Both are good. However, for convenience and flexibility, I would recommend STATA. Oher options are R and MATLAB

In Mackey-Glass and other chaotic time series, shorter time delays are generally more important as they capture the underlying dynamics more accurately and prevent overfitting.

You can use econometric methods such as regression analysis, Vector Autoregression (VAR), or Granger causality tests to analyze how macroeconomic variables affect REIT index returns.

After performing a stationary test for your data analysis using the ADL model, you have found that the dependent variable and 6 out of 8 independent variables are stationary only in their differences, while the other two are stationary in levels. In this scenario, you can proceed with modeling your data using an Autoregressive Distributed Lag (ADL) model.

Adam Tomko Moges Mengstu Kassaw Srk Haqbin Benjamine Gaspar Miku The ADL model is suitable for situations where variables exhibit different stationarity properties, such as some being stationary in levels and others in differences.The ADL model allows for the inclusion of lagged values of both the dependent and independent variables, accommodating the mixed stationarity properties of your variables. By incorporating lagged values of the variables that are stationary in differences, you can capture the short-term dynamics and relationships in your data. At the same time, including the variables that are stationary in levels enables you to account for the long-term equilibrium relationships.

This approach aligns with the flexibility of the ADL model, which can handle variables with diverse stationarity characteristics, making it a suitable choice for your data analysis scenario. By appropriately specifying the model with the lagged terms of the variables based on their stationarity properties, you can effectively capture the dynamics and relationships within your dataset. Adam Tomko

Wavelet denoising is used in radar and sonar. e.g.

Use correlation or regression methods

Francisco Fatas-Villafranca

thank you vey much. Your information was vey useful. I have downloaded all three papers.

Best,

Yoshinori

Hi Cem Avcı

Some information from RG:

https://www.researchgate.net/post/How_to_assess_synchrony_between_time_series

https://www.researchgate.net/post/Multiple-cross-correlation-function

However, as you said, the observations are few …

MRI datasets typically result in high-resolution three-dimensional images representing anatomical structures. These images are often stored in formats such as DICOM (Digital Imaging and Communications in Medicine) or NIfTI (Neuroimaging Informatics Technology Initiative). fMRI datasets produce time-series data representing changes in brain activity over time. These data are often stored in formats compatible with neuroimaging software packages, such as NIfTI, Analyze, or MINC (Medical Imaging NetCDF). fMRI data can be conceptualized and analyzed both as images and time-series. The choice of representation depends on the specific research question and analysis techniques being employed. For many analyses, researchers will use both approaches, leveraging the spatial information provided by the image-like representation and the temporal dynamics captured in the time-series data.

Yes, it is important to account for uncertainty Latent Class Growth Mixture Models (LCGMM), these models are used to identify homogeneous subpopulations within a heterogeneous population, and it also help to identify meaningful groups or classes of individuals.

Hello Zainab Khan

I hope you are doing well. I have previously worked in something similar, so one input I can give you is that when analysing time series data, it is vital to try and standardize the whole data from the first to the last entry. Even though the vegetation indices cancel out many annoying things, it is still very robust to use atmospheric corrections for all the data as you want all other variables (other than your target index) to be as less varying as possible.

I know this is vague but I'll need more information from you to discuss this any deeper.

Best wishes.

A late answer maybe for other people. To the best of my knowledge, the Pettitt test hasn't been extended for more than one breakpoints: the extension is challenging due to the nonparametric nature of the test. Model-based statistical techniques may be more relevant to handle complex situations (e.g., with an unknown number of changepoints). In R, there are many packages dealing with this such as the changepoints and bcp packages. If anybody cares a Bayesian method, one choice is the Rbeast package I recently developed; it has been used in dozens of fields and is available in R, Python, and Matlab: https://github.com/zhaokg/Rbeast.

Dear Muhammad Shahid

I also have the same question. If you found the way can you please guide me.

I really appreciate your help.

Kind regards,

Sineka

Through the following.

In time series data analysis, a concise approach involves several key steps:

1. Data Exploration:

- Understand the characteristics of the time series.

- Check for trends, seasonality, and outliers.

2. Data Preprocessing:

- Handle missing values and outliers appropriately.

- Consider normalization or scaling if needed.

3. Visualization:

- Plot the time series to gain insights.

- Use tools like line plots, histograms, or box plots.

4. Feature Engineering:

- Extract relevant features, such as rolling statistics or lag features.

- Consider transformations for stationarity.

5. Model Selection:

- Choose an appropriate model based on characteristics (ARIMA, SARIMA, LSTM, etc.).

- Split data into training and testing sets.

6. Training:

- Train the selected model on the training set.

7. Evaluation:

- Evaluate model performance on the test set.

- Use metrics like Mean Absolute Error (MAE) or Root Mean Squared Error (RMSE).

8. Optimization:

- Fine-tune model parameters if needed.

- Consider ensemble methods or hybrid models.

9. Prediction:

- Use the trained model to make predictions on new data.

10. Validation:

- Validate results against actual outcomes.

- Adjust the model or methodology if necessary.

Each work involves implementing these steps using statistical and machine learning techniques, and it often requires adapting strategies based on the specific characteristics of the data. It's crucial to stay mindful of the context and the goals of the analysis throughout the process.

Hi,

Your approach is good. Use ARIMA for trend analysis and panel data regression for regression analysis, ensuring to check for stationarity and autocorrelation in your time series data.

Hope this helps.

Whether your T is large enough for your analysis depends upon several factors, especially on the model specified and the estimator employed. However, values of T around 30 can be considered sufficiently large in many empirical applications.

Statistical tool or test can be used for dose-response curve with time component is the nonlinear regression. You can use nonlinear regression to quantify the fold shift in the dose and its confidence interval, and to compute a P value that tests the null hypothesis that there was no shift.

That method is only applicable when the number of means is greater than 5, otherwise you must use multiple means comparison methods such as Dunca, Tukey, Newman-Keuls, etc. However, the statistical design issues related to dose-ranging studies are not always keenly understood and a poorly designed study can be costly for later development.

I suggest that you employ an ARCH/ARMAX method where the ARCH component models the conditional variance of the error term, the ARMA component models the autoregressive nature of your data, and the X component models the effects of the exogenous variables.

Here is a link to a recent application of the method:

In time series analysis, whether to use a multiplicative model or an additive model depends on the characteristics of the underlying data. Here are some considerations that might make the multiplicative model more appropriate:

In contrast, an additive model is more appropriate when the seasonal variations and other components are consistent in magnitude regardless of the level of the series. The choice between a multiplicative and additive model is not always clear-cut, and sometimes experimentation with both models is necessary to determine which one provides a better fit to the data. Additionally, the nature of the data and the specific characteristics of the time series should guide the choice of the appropriate model.

To export time series data from Google Earth Engine to ASCII format for use in TimeSat, you can follow these general steps:

Now, you have an ASCII file that can be used as input to TimeSat. Adjust the steps based on your specific requirements and the dataset you are working with.

I have used a MATLAB code to extract it from IONEX files, added coordinates and time column and save it to excel file, if this is what you want to do, I can provide you with that code.

Hi Nivaj,

please take a look at the following link:

I see that the author uses time series analysis

Best,

Apostolos

Hi Fatima,

please take a look at the following link: https://davegiles.blogspot.com/2011/04/testing-for-granger-causality.html

I think that it will be very helpful (especially if you use the Eviews software). It explains the procedure step by step.

The test does not distinguish between long and short-run effects.

Saheed Busayo Akanni Probably because the range is too large to be shown easily and clearly on a linear axis. Logarithms will compress the data. Consider trying to show $1, $100, $1 million, $1 billion dollars on a linear axis. Consider the same situation with logarithms.

What is your question?

Your efforts are appreciated

No Viktor Zadachyn it is possible provided you have a related monthly time series with the proper periodicity. Of course it will differ from the true (unknown) monthly time series.

@Davood-Omidian yes i understand your answer, it is very important, I will try it, thank you very much

Best regards

Dear Saba Moshtaghi,

The availability of historical climate data can fluctuate based on the specific location and the particular parameters of interest. To ensure accurate data retrieval, it is crucial to possess the precise name and geographic coordinates, including latitude and longitude, of the lake in question.

Numerous government agencies, such as the National Weather Service, diligently gather and manage climatic data for various locations, including lakes. Additionally, you can explore organizations and websites that offer access to climate data, such as NASA's Earthdata Search and the World Meteorological Organization.

It is worth noting that certain datasets may necessitate permissions or subscriptions for access, requiring initial registration. Furthermore, depending on your research objectives, it may be advisable to consider data from at least three nearby weather stations and compute an average. This becomes especially critical when dealing with rainfall data, as precipitation patterns can vary significantly both temporally and spatially.

humble regards,

Yes, you can use ARDL for time series data analysis even if some of the variables are in absolute values and some are in percentage terms. ARDL (Auto Regressive Distributed Lag) is a method that can be used for modeling the relationship between two or more time series variables. It is particularly useful when the variables are non-stationary, meaning that they have a trend or a seasonality component that needs to be accounted for. ARDL can handle both stationary and non-stationary variables, and it can also handle mixed orders of integration, such as I(0) and I(1)

Therefore, you can use ARDL to model your time series data that contains both absolute values and percentage terms. @ritu

There could be many options for detrending or fitting a trend. In Python, one choice is to fit a nonlinear trend and get the remainder as the detrend signal using. One such tool is this package at https://pypi.org/project/Rbeast/ where in the second example, the trend is removed, leaving only the seasonal variation. But depending on your use scenarios, a simple linear regression may be sufficient for the detrending purpose.

Many options are possible. If you are using R, let me borrow from the time series CRAM task view:

Change point detection is provided in strucchange and strucchangeRcpp (using linear regression models) and in trend (using nonparametric tests). The changepoint package provides many popular changepoint methods, and ecp does nonparametric changepoint detection for univariate and multivariate series. changepoint.np implements the nonparametric PELT algorithm, changepoint.mv detects changepoints in multivariate time series, while changepoint.geo implements the high-dimensional changepoint detection method GeomCP. Factor-augmented VAR (FAVAR) models are estimated by a Bayesian method with FAVAR. InspectChangepoint uses sparse projection to estimate changepoints in high-dimensional time series. Rbeast provides Bayesian change-point detection and time series decomposition.

Of these, the Rbeast package is maintained by me and also available in Matlab and Python. If interested, see more infor at https://github.com/zhaokg/Rbeast.

Check Google

Dr Biswajit Sarangi thank you for your contribution to the discussion

Dr Biswajit Sarangi thank you for your contribution to the discussion

Dear Dr. Mohammad Imam Your comment does not carry a semantic load, it looks like a general instruction for creating a package for Python

There are several ways to convert a daily series into a monthly one. Popular methods include

What you do now depends on the nature of your monthly variable and the process you are trying to capture. Look at the economics or finance of the process. For example, if you think that your monthly variable is dependent on the volatility and level of your daily series, you might consider using one of 1 to 4 for the level and 7 or 8 for the volatility. There are many combinations that might be useful but that is up to you. It may also be necessary to log transform your variables before analysis.

It depends on the nature of your study and the data you have. Can you please provide more details about your study?

thanks Hamid

this dataset is befit. however, I would like to find a dataset on the case of a house that contains several rooms (bedrooms or other places in the house like bathroom, kitchen, etc.)

yours

To split your data into training and test sets for predicting stock prices using pre-COVID and current COVID periods, consider using a time-based approach. Allocate a portion of data from pre-COVID for training and the subsequent COVID period for testing, ensuring temporal continuity while evaluating predictive performance.

Some patent indicators that can be analyzed in the context of time series are:

- Patent applications and grants

- Patent families

- Patent citations

- Patent classifications

#Remote Sensing of Environment (Recommended) (link: https://www.sciencedirect.com/journal/remote-sensing-of-environment)

#Environmental Remote Sensing (link: https://www.mdpi.com/journal/remotesensing/sections/environmental_remote_sensing)

#Journal of Remote Sensing (in partnership with science) (link: https://spj.science.org/journal/remotesensing)

Access the link and find out if there is any other: https://www.gisvacancy.com/remote-sensing-journals/

The problem should be treated in a deeper manner:

(1) How large is the sample size, and what is the share of missing values?

(2) What is the type of your time series? In which field is your approach?

(3) What is the time measure? Intra-day?, dayli, monthly, sub-annual, etc.?

(4) Where are the missing values situated? At the begining, at the end of the time serie?

After you answer on these questions you may search for an appropriate method of inputation, like Chuck and Mirajul already mentioned.

Use normalization techniques such as min-max normalization or z-score normalization. Another method is to use feature scaling techniques such as standardization or normalization.

One possible solution is to increase the amount of memory allocated to ImageJ. You can do this by going to Edit > Options > Memory & Threads and increasing the maximum memory allocation. Another solution is to try closing other programs that are running on your computer to free up memory.

I am afraid that the suggestions given by Chuck A Arize do not answer the question. You would like a kind of t-test but the problem is that the two time series are not random samples. If the series are stationary, it can probably be done but taking the autocorrelation into account but I don't know how. If the time series are not stationary, there is little hope. Good luck

One way to estimate the length of each period or seasonality is to find the frequency corresponding to the maximum of the spectral density. However, the spectrum at low frequencies will be affected by trend, so you need to detrend the series first.

Another way is to use functions such as seasonal_decompose and STL (Python statsmodels package) or models like SARIMA that have a period or cycle parameter that indicates ‘the period of the series’ used (period, seasonal, etc)

A time series regression model will have the form

y_t = \beta_1 + \beta_2 x_{2t} + \beta_3 x_{3t} + \dots + \beta_k x_{kt} +u_t

where y is time series of the independent variable and

x_2,x_3,\dots,x_k are explanatory time series variables. (Certain conditions must be satisfied if the coefficients are to be estimated. There are additional problems to be addressed if any of the series are non-stationary)

While I have seen studies with 30 observations of each variable that have provided useful results generally considerably more observations are usually required.

Look at your data. As far as I can tell you do not have the necessary time series independent variable to do a time series regression. Perhaps I still do not understand.

Dear Fadhil Abdulabbas Abidi ,

An Artificial Neural Network (ANN) is an information processing paradigm that is inspired by the way biological nervous systems, such as the brain, process information. The key element of this paradigm is the novel structure of the information processing system. It is composed of a large number of highly interconnected processing elements (neurones) working in unison to solve specific problems. ANNs, like people, learn by example. An ANN is configured for a specific application, such as pattern recognition or data classification, through a learning process.

Regards,

Shafagat

If a time series is stationary (constant mean and variance), then we can model the volatility with GARCH. GARCH models are used to model the volatility of time series data that is not stationary. However, if the time series is stationary, then we can use GARCH models to model the volatility of the time series.

You can use non-parametric tests to compare two marine stations over 2 years. Non-parametric tests are valid for both non-Normally distributed data and Normally distributed data. Some examples of non-parametric tests include the Mann-Whitney U test and the Wilcoxon signed-rank test. The Mann-Whitney U test is used to compare two independent groups. The Wilcoxon signed-rank test is used to compare two dependent groups.

This is incorrect. The ADF test (or its variants) makes no such assumption. In fact, the ADF test is used to do a preliminary assessment of the stochastic property of a given time series, namely, trend-stationary vs. differenced-stationary vs. both.

Prof. Ahking

That method for detrending precip. (avoiding negative precip., conserving dry days as dry days, and conserving the mean) has been published, as part of the TRANSLATE project in: https://www.frontiersin.org/articles/10.3389/fclim.2023.1166828/full . Please cite that source if you use this method. It's a good, simple method!

Hi,

Just to mention (refere to wikipedia), the Durbin–Watson statistic, while displayed by many regression analysis programs, is not applicable in certain situations. For instance, when lagged dependent variables are included in the explanatory variables, then it is inappropriate to use this test. Durbin's h-test (see below) or likelihood ratio tests, that are valid in large samples, should be used.

Hamid

Akrivi Alexandraki , Even annual data might have a correlation in my sense. For example, if your data has a clear trend over time it may show a correlation. Autocorrelation is the relation between lagged values. So, seasonality is not mandatory for autocorrelation.

One way to deal with both negative and positive time series in a single econometric model is to use the absolute value of the time series. Another way is to use logarithmic transformation. The logarithmic transformation can be used when the data contains both positive and negative values. The log transformation can be applied to the absolute value of the data plus one.

Seasonal-Trend decomposition using LOESS (STL) is a time series decomposition technique that separates a time series into three components: trend, seasonal, and remainder. The trend component captures the long-term changes in the data, the seasonal component captures the repetitive patterns that occur within a year, and the remainder component represents the random variation or noise in the data.

The STL method uses a non-parametric technique called LOESS (Locally Estimated Scatterplot Smoothing) to estimate the trend and seasonal components of the time series. LOESS uses a moving window to estimate the local polynomial that fits the data, with the window size varying depending on the density of the data points. This allows LOESS to capture complex nonlinear relationships in the data, including seasonal patterns.

Therefore, STL does remove both trend and seasonality from a time series by decomposing the time series into these two components, leaving only the remainder component which represents the random variation or noise in the data. The remainder component is often easier to model and analyze than the original time series since it has had the trend and seasonal components removed. However, it is important to note that the accuracy of the decomposition can be influenced by the choice of parameters, such as the window size used in the LOESS smoothing, and may require some tuning based on the specific characteristics of the data.

Thank you very much for the advice Mr Samuel Oluwaseun Adeyemo.

Hello , when do we use the granger casualty test and the cointergration test

It is not the fact that the data are time series which is important. What is important is the presence of autocorrelation. For example, if your data are about the composition of the soil along a road, for instance, every 10 meters, you will also face autocorrelation despite the fact that the data are not time series. It is known how to handle autocorrelation for regression data but it adds difficulties and it requires typically longer samples.

Thank you ,

But here is the bond test result: an evidence of cointegration among the variables ....

Yes, it is possible to develop regression models for time series data with breaks. Remember that this is also known as interrupted time series analysis, which uses regression techniques to model variations in the level or trend of a time series following an intervention or event. These models can be utilised to determine the impact of  changes on a time series variable.

It is essential, however, to ensure that the time series breaks are well-defined and supported by theory or empirical evidence. In addition, the validity of the regression models may be depending on the assumptions made regarding the nature and timing of the breaks as well as the potential confounding variables that may influence the outcome variable. Therefore, models require thorough consideration and validation to ensure accurate results.

covid <- c(171, 0)

res <- chisq.test(covid, p = c(1/2, 1/2))

Chi-squared test for given probabilities data: covid X-squared = 171, df = 1, p-value < 2.2e-16

If you have a time dependent set of data I suggest looking at regression form ANCOVA. David Booth also see Marie Davidian work on longitudinal models.

Hi there,

1 - In this context, phase stability refers to the consistency of the phase values acquired by the InSAR system over time for a particular pixel. A pixel with high phase stability will have phase values that are consistent across multiple acquisitions, while a pixel with low phase stability will have phase values that vary significantly over time. Phase stability is an important factor in InSAR analysis as it directly affects the accuracy of the deformation measurement.

2 - The amplitude dispersion across a time series of InSAR images can affect the phase's standard deviation by introducing noise to the system. When the amplitude of the signal varies significantly over time, it can result in a loss of coherence between the two acquisitions being compared, leading to increased phase noise and decreased phase stability. The selection of persistent scatterers candidates (PSCs) based on low amplitude dispersion index thresholding is done to ensure that the selected pixels have high coherence, which in turn ensures higher phase stability.

3 - Uncompensated propagation disturbances such as atmospheric phase contribution and satellite orbital position inaccuracies can significantly affect the phase stability of InSAR images. The atmospheric phase contribution is particularly problematic, as it can introduce significant phase noise into the system. This noise can be mitigated using various techniques, such as using a weather model to correct for the atmospheric contribution, or by using differential InSAR techniques that cancel out the atmospheric phase contribution. Satellite position inaccuracies can also affect the phase stability, particularly if the satellite's orbit is not accurately known. This can result in phase errors that can be corrected using various techniques, such as using GPS data to refine the satellite orbit.

Dear Lea,

if I understand your query correctly, you want to analyze which factors drive temperature variability on longer than diurnal and tidal timescales in this stream. To smooth out this high-frequency "noise" and detect the signals you are actually after, I'd suggest to calculate 48-hour or even longer averages. These composite averages should yield you a clearer picture of hydrographic temperature variations above the daily/tidal, but still below the seasonal scale. You should, however, also take into account short-term ambient (atmospheric) weather variability, which may produce profound fluctuations in water temperature, especially in the upper layers of the water column. Such variations induced by weather variability are still below the seasonal time scale, but it may sometimes be difficult to discern or disentangle the two signals, especially in transition periods between two seasons or during long-lasting weather patterns. Do you collect meteorological data alongside your potamological parameters?

Hope this helps a bit.

Best,

Julius

Thanks again Nderitu Githaiga for your suggestion.

Real-time data sources for power systems and electrical machines can include sensors that measure parameters such as voltage, current, temperature, and vibration. These sensors can be installed directly on the machines or on the surrounding equipment, and can send data to monitoring and control systems in real-time. Other sources of data can include digital meters, protective relays, and SCADA (Supervisory Control and Data Acquisition) systems. All of these sources can provide important information for monitoring the health and performance of power systems and electrical machines.

As you have two raster images, you can export those raster images with the same pixel size and convert those images into vectors (point or polygon shapefiles). Then you can perform the overlay analysis (Union or intersection for a particular crop) of two vectors and then you will be able to get the NDVI Values. If you perform overlay analysis for the point shapefile then you will get NDVI values for each pixel of the raster image.

Dear Prof

Papers relating brain temperature dynamics, it's available sir