Nowadays, Machine Learning (ML) can serve as one of the solutions to accelerate the process of decision-making in forecasting daily stock market price movements. Nonetheless, inadequate number of research and lack of extensive data analysis using various ML models had limit the investors to appreciate the efficiency and capability of these models. Previous studies usually concentrate on the forecasting stock index or selecting a few stocks with restricted features. Therefore, this study focused to contribute on evaluating different algorithm models such as traditional ML and deep learning models with big stock data of multiple parameters from selected companies in Bursa Malaysia. The three traditional ML selected includes Logistic Regression (LR), Support Vector Machine (SVM), and Extreme Gradient Boosting (XGB), while another three deep learning models selected are Deep Belief Network (DBN), Multilayer Perception (MLP), and Stacked Auto-Encoder (SAE). By setting the ML algorithms and their parameter along with using Walk-Forward Analysis (WFA) method, the algorithm design of trading signal was evaluated based on two groups of evaluation indicators, namely directional and performance. Comparative analysis of evaluation indicators for all trading algorithms has been assessed and discussed. For stock trading in Malaysian stock market particularly, the experimental results of this study demonstrate that deep learning models have better performance in directional evaluation indicator compared to traditional ML in time series forecasting. However, traditional ML models are more efficient than deep learning in performance evaluation indicators in terms of profitability and risk assessment.

In the modern financial market, investors have to make quick and efficient investment decisions. The problem arises when the investor does not know the right tools to use in investment decision making. Different tools can be implemented in trading strategies to predict future stock prices. Therefore, the primary objective of this paper is to analyse the performance of the Geometric Brownian Motion (GBM) model in forecasting Nestle stock price by assessing the performance evaluation indicators. To analyse the stocks, two software were used, namely Microsoft Excel and Python. The model is trained for 16 weeks (4 months) of data from May to August 2019 and 2020. The simulated sample is for four weeks (1 month) which is for September 2019 and 2020. The findings show that during the Pandemic Covid-19, short-term prediction using GBM is more efficient than long-term prediction as the lowest Mean Square Error (MSE) value is at one week period. In addition, the Mean Absolute Percentage Error (MAPE) for all GBM simulations is highly accurate as it shows that MAPE values are less than 10%, indicating that the GBM method can be used to predict Nestle stock price during an economic downturn.

Nowadays, Machine Learning (ML) plays a significant role in the economy, especially in the stock trading strategy. However, there is an inadequate extensive data analysis using various ML methods. Previous findings usually focus on the forecasting stock index or selecting a limited number of stocks with restricted features. Therefore, the contribution of this paper focused on evaluating different supervised learning algorithms, namely Logistic Regression (LR), Support Vector Machine (SVM), and Extreme Gradient Boosting (XGB), on a big dataset from 28 stocks in Bursa Malaysia. By setting their parameter along and using Walk-Forward Analysis (WFA) method, the trading signal was evaluated based on Accuracy Rate, Precision Rate, Recall Rate, and F1 Score. For stock trading strategies in Malaysia in particular, the findings of this study show that SVM has a better performance compared to LR and XGB in time series forecasting. The ML algorithms have values ranging from 53% to 66% for Accuracy Rate (AR), Recall Rate (RR), and F1 Score (F1). In addition, SVM has the highest Precision Rate (PR) of 73% among the ML algorithms.