Data: Linear Regression Modelling of Population and GDP using Adaptive Learning Rate Optimization Algorithms

Adaptive Learning Rate Optimization Algorithms (ALROA) are used in solving machine and deep learning problems. This study aims to adopt these algorithms to estimate the coefficients of traditional linear regression models between global population and GDP by countries for the Year 2019. Algorithms such as Stochastic Gradient Descent (SGD), Stochastic Gradient Descent with Momentum (SGD with Momentum), Nesterov Accelerated Gradient Descent (NAG), Modified Root Mean Squared Propagation (Modified RMSprop), and Modified Adaptive Moment Estimation (Modified Adam) were used in this study. A learning rate of 0.01 and exponential decay rates of 0.9 and 0.999 were used for the first and second momentum. Half Mean Square Error (HMSE) was used as the loss function while Root Mean Square Error (RMSE) was used to rank the algorithms in order of accuracy. Keywords: Cost Function, Modified Adam, Gradient Descent, RMSE自适应学习率优化算法（Adaptive Learning Rate Optimization Algorithms, ALROA）可用于求解机器学习与深度学习问题。本研究旨在采用此类算法，针对2019年各国的全球人口与GDP之间的传统线性回归模型的系数开展估计工作。本研究选用的算法包括随机梯度下降（Stochastic Gradient Descent, SGD）、带动量的随机梯度下降（Stochastic Gradient Descent with Momentum, SGD with Momentum）、内斯特罗夫加速梯度下降（Nesterov Accelerated Gradient Descent, NAG）、改进型均方根传播（Modified Root Mean Squared Propagation, Modified RMSprop）以及改进型自适应矩估计（Modified Adaptive Moment Estimation, Modified Adam）。本研究设置初始学习率为0.01，并为一阶矩与二阶矩分别配置指数衰减率0.9与0.999。本研究以半均方误差（Half Mean Square Error, HMSE）作为损失函数，以均方根误差（Root Mean Square Error, RMSE）为评价指标，按照算法的准确率对其进行排序。关键词：代价函数（Cost Function）、改进型自适应矩估计（Modified Adam）、梯度下降（Gradient Descent）、均方根误差（RMSE）