Applying Multi-Task Deep Learning Methods in Electricity Load Forecasting Using Meteorological Factors

Abstract

The steady rise in carbon emissions has significantly exacerbated the global climate crisis, posing a severe threat to ecosystems due to the greenhouse gas effect. As one of the most pressing challenges of our time, the need for an immediate transition to renewable energy is imperative to meet the carbon reduction targets set by the Paris Agreement. Buildings, as major contributors to global energy consumption, play a pivotal role in climate change. This study diverges from previous research by employing multi-task deep learning techniques to develop a predictive model for electricity load in commercial buildings, incorporating auxiliary tasks such as temperature and cloud coverage. Using real data from a commercial building in Taiwan, this study explores the effects of varying batch sizes (100, 125, 150, and 200) on the model’s performance. The findings reveal that the multi-task deep learning model consistently surpasses single-task models in predicting electricity load, demonstrating superior accuracy and stability. These insights are crucial for companies aiming to enhance energy efficiency and formulate effective renewable energy procurement strategies, contributing to broader sustainability efforts and aligning with global climate action goals.

1. Introduction

1.1. Background and Motivation

The world has been facing increasingly severe challenges of climate change, with frequent occurrences of extreme weather events, such as extreme heat, intense storms, and persistent droughts, causing significant impacts on human society and ecosystems. Among these challenges, greenhouse gas emissions are recognized as one of the primary causes, leading to global temperature rise and triggering extreme climatic phenomena. The issue of carbon emissions has become a focal point of global concern, necessitating proactive actions from humanity to reduce carbon emissions and address the challenges brought about by climate change. Climate change not only affects land, food, and water resources but also leads to social unrest and economic turmoil.

Renewable energy has emerged as a critical strategy to address the climate crisis. Renewable energy sources, such as solar, wind, and hydropower, have lower carbon emissions and are sustainable energy sources with less environmental impact. Therefore, promoting the use of renewable energy is a key measure in achieving carbon reduction targets and tackling climate change. According to [1], the COVID-19 pandemic and the global energy crisis have led to a significant increase in investments related to renewable energy worldwide. Figure 1 shows the global distribution of investments in fossil fuels and green energy from 2015 to 2022. It is evident from Figure 1 that investments in green energy have been steadily increasing since 2020, while investments in fossil fuel energy experienced a sharp decline during the COVID-19 outbreak in 2020 compared to 2019. This indicates the growing emphasis on utilizing and developing renewable energy globally. Such a shift serves as a crucial driving force and direction for achieving energy transition and reducing carbon emissions.

Electricity is a vital component in modern life, as well as for the operations of businesses, leading to a continuous increase in global electricity demand. However, due to the complexity and cost of electricity production processes, the power industry faces significant challenges in meeting this surging demand. The production, transmission, and distribution of electricity are confronted with formidable obstacles. Consequently, to meet the ever-growing electricity demand, the effective management of the power grid is essential [2].

With an increasing demand for sustainable energy, solar energy represents a cost-effective and abundant renewable resource that is readily harnessable, even under overcast conditions. Given its pivotal role in powering future energy needs, renewable energy has become integral to the global electrical power supply. In 2021, the Ministry of Economic Affairs (MOEA) introduced the “Regulations for the Management of Setting up Renewable Energy Power Generation Equipment of Power Users above a Certain Contract Capacity”, which is commonly referred to as the “High Electricity User Clause”. According to this regulation, large electricity consumers with a contract capacity of 5000 kW or higher are obligated to procure or install 10% of their energy from renewable sources within a five-year buffer period [3].

Commercial buildings play a significant role as sources of energy consumption, exerting a crucial impact on carbon emissions and energy usage. Specifically, ref. [4] highlighted that integrating comprehensive actions across various departments within enterprises, with particular emphasis on building energy consumption, constitutes a responsive action to climate change worldwide. Commercial buildings significantly contribute to carbon emissions and are recognized as the primary contributors to energy consumption, making them a key factor in climate change. As a result, countries worldwide are currently implementing various critical measures to develop energy-efficient buildings or utilize renewable energy sources to reduce carbon footprints and mitigate the effects of climate change on commercial buildings.

One of the power sources for commercial buildings is renewable energy, which can substantially reduce carbon emissions and minimize environmental impacts. Solar panels and other climate-friendly energy sources are effective in decreasing carbon emissions and can be installed in commercial buildings to generate renewable energy, particularly since the peak demand for electricity in these buildings occurs during daylight hours. By integrating solar panels, commercial buildings can substitute their existing power supply with renewable energy sources.

However, to effectively promote the adoption of renewable energy, a comprehensive understanding of energy usage is essential. The electricity demand in buildings correlates closely with climate-related factors. Predicting electricity consumption in commercial skyscrapers becomes a significant research topic, as it not only assists businesses in planning and managing energy consumption more efficiently but also provides valuable guidance to enterprises seeking to procure renewable energy.

1.2. Research Objectives

With the increasing awareness of environmental, social, and governance (ESG) considerations and the imperative to fulfill corporate social responsibility (CSR), corporations are showing a growing demand for green electricity. Addressing this demand necessitates effective energy management and planning to optimize energy efficiency. The analysis of electricity load offers valuable insights into the electricity consumption patterns of enterprises, identifying potential areas for enhancement.

Hence, the main objective of this paper is to propose a deep learning model based on multi-task learning that is specifically tailored for forecasting electricity load in commercial buildings. The principal task involves accurate electricity load forecasting, while two auxiliary tasks, namely, temperature forecasting and cloud cover forecasting, are integrated to enhance the overall performance of the primary task. By concurrently forecasting electricity load, temperature, and cloud cover, this research aims to accomplish the following objectives based on the acquired outcomes:

The first is to support companies in defining the appropriate contract capacity based on electricity load demand analysis, thereby ensuring an optimal balance between energy costs and the synchronization of supply and demand, ultimately maximizing benefits.

The second is to empower managers to obtain a comprehensive understanding of electricity consumption patterns, enabling the establishment of optimal green energy procurement plans and leading to more effective energy procurement strategies.

The third is to apply the derived findings to practical energy management in commercial buildings, facilitating comprehensive assessments and analyses of energy usage and identifying potential for energy-saving and improvement strategies.

The final objective is to provide the obtained results and valuable insights as a reference for subsequent academic research and practical applications in the field.

The proposed model aims to enhance the accuracy and efficiency of electricity load forecasting by incorporating multiple tasks into the learning process. Following the model’s development, an empirical analysis will be conducted to evaluate its performance and capabilities.

The experimental results of this study reveal that this can significantly increase the proposed model’s performance and reduce the variance in the prediction results. The results from the empirical analysis will provide valuable insights and implications for electricity load forecasting using multi-task deep learning techniques, as well.

The remaining sections of the paper are organized as follows: Section 2 reviews the relevant literature and methodologies adopted in electricity load forecasting and meteorological factors. Section 3 will explain the method of solving the problem by utilizing multi-task deep learning techniques. In Section 4, the research findings will be discussed, and finally, the management insights and conclusions will be described in Section 5.

2. Literature Review

This section aims to conduct a literature review on the subject of electricity load forecasting and explore the correlation between electricity load and various meteorological factors. Additionally, it compiles the literature on single-task and multi-task deep learning methods. Finally, based on the findings from the literature review, a summary is provided to serve as the foundation for the subsequent research methodology.

2.1. Electricity Load Forecasting

Electricity load forecasting plays a crucial role in the electricity system planning and decision-making processes of enterprises. The purpose of electricity load forecasting is to predict future electricity consumption at one or multiple locations within the power system network [5]. It is widely acknowledged that electricity load is dynamic, and therefore, no model has been able to provide precise predictive results, making electricity load forecasting inherently uncertain. Furthermore, there is a lack of clear and standardized definitions regarding the classification of electricity load forecasting scopes [6]. The accuracy of electricity load forecasting is of paramount importance for the planning of power generation facilities’ expansion and maintenance cycles. If the forecasting results underestimate electricity consumption, it may lead to power shortages and blackouts, causing inconvenience and economic losses to consumers. Conversely, overestimation of the forecasted load would result in energy waste and unnecessary cost expenditures. Therefore, precise electricity load forecasting contributes significantly to ensuring the stability and economic efficiency of the power system.

Regarding the classification of electricity load forecasting, previous scholars have categorized it into short-term, mid-term, and long-term load forecasting based on different time ranges and operational requirements. Various types of electricity load forecasting are applied for different purposes, providing valuable information for diverse operational decisions [7,8,9,10].

Ref. [10] mentioned that the short-term electricity load forecasting process typically considers key factors, such as seasonality, weather-related information, and historical data. Seasonality factors encompass the impact of heating, ventilation, and air conditioning systems on electricity demand. Weather-related information includes temperature, cloud cover, humidity, and wind speed, as weather conditions significantly influence electricity demand. On the other hand, historical data consist of past hourly electricity load information. The primary objective of short-term electricity load forecasting is to assist management teams in making timely, informed, and accurate decisions for the operation of the power system on a daily basis.

Medium-term load forecasting, as a critical component of electricity load forecasting, focuses on predicting electricity consumption over a medium-term time horizon, typically ranging from a few days to several weeks or months. Unlike short-term load forecasting, which deals with immediate operational decisions, and long-term load forecasting, which involves strategic planning, medium-term forecasting addresses the mid-range planning needs of power utilities and energy market participants. The primary use of medium-term load forecasting is for capacity planning and resource allocation. Power utilities and energy providers require accurate medium-term load forecasts to make informed decisions about the expansion or contraction of their power generation capacities, ensuring that they can meet the future electricity demand efficiently. This planning process is essential to maintaining the reliability and stability of the electricity grid, avoiding potential shortages, and preventing excessive investment in redundant generation capacity.

Many studies have explored the application of different methods and techniques in electricity load forecasting. Some research focuses on statistical models, such as autoregressive integrated moving average (ARIMA) and seasonal autoregressive integrated moving average (SARIMA) models, for long-term and short-term load forecasting [11,12]. Zhao et al. [13] summarized regression-based methods to predict building electricity consumption, and the methods regarding machine learning techniques, such as support vector machine (SVM), artificial neural networks (ANNs), and random forest, to enhance prediction accuracy and adapt to various forecasting scenarios. In recent years, deep learning techniques, particularly long short-term memory (LSTM) [14,15] and convolutional neural network (CNN) have achieved significant advancements in electricity load forecasting [16,17,18]. The data source for the electricity load prediction algorithm mentioned above is usually the data on the energy management system (EMS). The advanced metering infrastructure system (AMI), which includes smart meters, can be combined with the EMS to provide accurate electricity load data at different time horizons and can be used with weather sensors, which can provide beneficial predictive characteristic data on electrical loads. In [19], a detailed report of various load forecasting methods conducted using conventional and smart meter information and forecasting for all types of loads has been widely discussed, including very short-term, short-term, medium-term, and long-term loads. In conclusion, electricity load forecasting is a crucial tool for ensuring the stable operation, efficient management, and sustainable development of the power system. By continuously referring to and applying the existing literature, we can continuously improve forecasting models and methods to meet the ever-changing energy demands and market challenges.

2.2. Meteorological Factors

In past research, many studies have investigated the impact of weather-related factors on building energy consumption and found correlations with solar radiation, direct radiation, temperature, relative humidity, wind speed, and atmospheric conditions [20]. Ref. [21] conducted a study that combined building energy simulation and sensitivity analysis to investigate the relationship between building energy consumption and meteorological factors, such as temperature, relative humidity, solar radiation, direct radiation, and wind speed. The research revealed that weather-related factors significantly exhibit seasonal and regional characteristics in their impact on building energy consumption. Different meteorological features contribute differently to building energy consumption, indicating the necessity to consider them separately in energy consumption analyses.

Cloud cover refers to the extent of the sky that is covered by clouds and plays a crucial role in meteorology. It is commonly measured using a ten-point scale, where higher values indicate a greater proportion of the sky covered by clouds. Cloud cover can be quantified on a scale from 0 to 10, where a higher value indicates a greater number of clouds and complete cloud coverage in the sky. Conversely, a value of zero indicates a clear sky with no visible clouds. When clouds are present above a solar panel, they cause the scattering of solar irradiance, resulting in a decrease in the amount of irradiation that reaches the solar panel. Therefore, there is an inverse relationship between cloud cover and power generation, implying that power generation tends to decrease as the number of clouds increases [22,23]. Clouds have a limited impact on cooling temperatures during the night, primarily because the temperature is naturally lower. However, on cloudy nights, air circulation is impeded, leading to higher nighttime temperatures, which in turn leads to an increased demand for cooling appliances like air conditioners [24,25]. As a result, electricity consumption rises during these periods. Additionally, during the daytime, the presence of denser cloud cover reduces the intensity of sunlight, causing outdoor temperatures to drop. Consequently, electricity usage decreases due to reduced cooling demands. However, it is worth noting that this reduction in energy consumption might be offset by an increase in lighting usage during cloudy days, contributing to higher overall electricity consumption.

As mentioned in the research objectives, the main focus of this study is to develop a multi-task deep learning model for electricity load forecasting. The main objective of this study is to use electricity load forecasting as the primary task to build an electricity load forecasting model using a multi-task deep learning approach. In this study, cloud cover will also be incorporated as an auxiliary task for prediction. This section will review the literature on the impact and correlation of weather-related factors, such as cloud cover and temperature, on electricity load forecasting.

Meteorological factors are recognized as influential contributors in electricity load forecasting, as they directly influence electricity consumption. Deep learning has become a widely adopted forecasting technique for effectively analyzing this complex data. Utilizing multi-layer neural networks, deep learning demonstrates the ability to autonomously learn and extract relevant features from extensive datasets. Moreover, the model can undergo continuous training and optimization, facilitating accurate forecasting and analysis. The subsequent section will provide an overview of the fundamental architecture of deep learning.

2.3. Deep Learning

In the domain of traditional machine learning methods, such as ANNs, a hidden layer is included, in addition to the input and output layers. ANN research has been limited to the investigation of one or two hidden layers [26]. During the training process, encountering underfitting difficulties poses significant challenges, hence ANN is also regarded as a shallow model.

Deep learning, which is derived from neuroscience and also known as deep neural networks (DNNs), represents a crucial branch of artificial intelligence and an advanced form of machine learning. Deep learning entails a multi-layer neural network architecture with multiple hidden layers, employing non-linear operations. This enables deep learning methods to efficiently capture the essence of highly non-linear functions with significant variations in a concise manner [27].

The composition of DNNs typically consists of an input layer, multiple hidden layers, and an output layer. Ref. [28] delves into the mechanisms of DNNs to gain a better understanding of how these models make predictions and decisions. At the beginning of a neural network is the input layer, which receives external input features, while the hidden layer serves as an intermediary between the input and output layers. Through the activation function, the input data are transmitted and processed across various layers until they reach the output layer, which produces the desired prediction for the target variable. The neurons within each layer are fully connected with those in the preceding layer. To achieve accurate forecasting, the network aims to minimize the discrepancy between the actual and forecasting results by employing different optimization techniques, loss functions, and weight adjustments. The structural representation is depicted in Figure 2 as provided.

Deep learning stands out as one of the prominent prediction methods due to its versatility and strong learning capabilities. Its depth implies that the neural network’s layered structure enables the propagation of data through the network to identify the optimal results and facilitate informed decision-making. Therefore, deep learning finds applications in diverse scenarios. In the subsequent section, the focus will be on single-task learning, which concentrates on a singular learning task. The conventional forecasting methods in electricity load, including statistical, machine learning, and deep learning methods, will be discussed. These methods all fall within the realm of single-task learning, and their respective limitations will be further elucidated.

2.4. Single-Task Learning (STL)

Research findings in the field of prediction have been mainly focused on learning individual specific tasks, which is a method known as single-task learning [29]. Single-task learning methods have been combined with statistical techniques, such as multiple regression and time series analysis, in the research on electricity load forecasting. Ref. [30] employed an integrated approach that combined multiple linear regression (MLR) and self-regression (SR) models to create a predictive model for forecasting the monthly electricity consumption in large public buildings. The multiple regression approach considered the linear combination and time series properties of various relevant factors. Ref. [31] analyzed time series data of household electricity consumption at various intervals, including daily, weekly, monthly, and quarterly. The investigation aimed to identify patterns and trends in the data. The authors found that the autoregressive integrated moving average (ARIMA) model provided the most accurate fit for forecasting electricity consumption. For forecasting over different time horizons, they segmented the forecast periods into monthly and quarterly intervals. Moreover, to optimize short-term forecasts at daily and weekly intervals, they utilized autoregressive moving average (ARMA) models. The results demonstrated the effectiveness of this approach for short-term electricity consumption forecasting.

While statistical methods are commonly utilized and relatively straightforward to apply, they possess certain limitations. One of these limitations arises from the necessity to adhere to specific assumptions, such as assuming linearity. However, concerning the intricate relationship between electricity load and influencing factors, particularly those linked to weather conditions, the association exhibits a complex and non-linear nature. Acknowledging the presence of nonlinearity is crucial, as it can substantially impact the accuracy of forecasts, and conventional statistical methods lack the flexibility required to effectively handle such non-linear challenges. Consequently, achieving enhanced predictive performance using these conventional models becomes arduous. The significance of nonlinearity on forecasting accuracy should be recognized, and it is important to acknowledge that traditional statistical methods are insufficiently flexible to address these non-linear challenges effectively [32,33].

The advent of machine learning techniques has revolutionized data analysis by relaxing the assumption of linearity in data distribution. ANNs, in particular, have garnered significant attention and adoption in load forecasting due to their capacity to capture complex non-linear relationships [34,35]. ANNs provide a versatile framework capable of effectively modeling intricate patterns and correlations, resulting in more accurate and nuanced predictions in load forecasting. Ref. [36] conducted research on short-term electricity consumption forecasting for buildings, and their findings demonstrated that the combination of ANNs with statistical analysis leads to substantial improvements in model selection and estimation. This integrated approach enhances the capabilities of neural network models and yields excellent predictive outcomes. Ref. [37] highlighted the significance of selecting appropriate ANN models while considering input variables like temperature, solar radiation, time, and historical electricity consumption data. By incorporating these factors into the forecasting process, the researchers achieved accurate predictions of electricity consumption in office buildings over a short-term period.

Deep learning is regarded as a groundbreaking advancement in forecasting methods, offering potential improvements in accuracy through its exceptional generalization capabilities. Unlike traditional approaches, deep learning models effectively capture complex patterns and relationships in data, enabling more accurate predictions across various domains. The ability of deep learning to generalize well to unseen data makes it a promising technique for enhancing forecasting accuracy, with researchers anticipating substantial improvements in forecast accuracy across a wide range of applications.

Ref. [15] aimed to enhance the accuracy of electricity consumption forecasts over extended periods. Their study demonstrated the effectiveness of deep learning techniques, specifically LSTM-based models, in generating reliable predictions of electricity consumption for both commercial and residential buildings over medium- to long-term time horizons.

Ref. [38] developed a hybrid forecasting model that combined a deep recurrent neural network (DRNN) and gated recurrent unit (GRU) to forecast the load demand of residential buildings at a 24 h interval. By capturing the time dependencies in the load data, the researchers aimed to model the temporal patterns and trends that influence electricity consumption. The experimental results showcased the superiority of the DRNN-GRU hybrid model over other methods, highlighting the effectiveness of integrating different deep learning architectures to leverage their complementary strengths and enhance the precision of load demand forecasting for residential buildings.

While deep learning techniques have demonstrated their effectiveness in electricity load forecasting, certain challenges remain to be addressed. A notable issue is the time-consuming nature of tuning deep learning models, primarily due to the complex training process involved. Researchers have acknowledged this concern, as highlighted in studies by [39,40]. Despite the promising results achieved by deep learning in load forecasting, efforts are being made to streamline and optimize the training process to make it more efficient and less time-consuming. Addressing these challenges would further enhance the practicality and usability of deep learning techniques in the field of electricity load forecasting.

2.5. Multi-Task Learning (MTL)

MTL is a machine learning approach aimed at simultaneously addressing multiple related tasks to enhance model generalization and prediction accuracy. In traditional single-task learning methods, each task typically requires the construction of an independent model, which can lead to the underutilization of data and potential overfitting when the sample size is limited. MTL achieves improved generalization by sharing information and knowledge across multiple related tasks. This means that when the model learns one task, it can simultaneously gain useful knowledge from other related tasks, thereby aiding in the improvement of prediction performance for each task. Ref. [29] introduced MTL as a mechanism designed to concurrently learn multiple interrelated tasks, with the primary objective of enhancing generalization by facilitating information sharing among these tasks. Furthermore, in contrast to single-task learning, multi-task learning exhibits the capacity to substantially improve learning efficiency and achieve accurate predictions for each individual task.

MTL can assist in solving complex and diverse real-world problems. In many application scenarios, different tasks are often closely interconnected and dependent on each other. For instance, in electricity load forecasting, electricity consumption may be influenced by various factors such as weather, holidays, and usage patterns. Constructing separate prediction models for each task may overlook the interrelated factors, whereas MTL effectively integrates them, resulting in more comprehensive and accurate load forecasting outcomes. MTL can also enhance the efficiency of model learning. By utilizing information from multiple related tasks during the learning process, the model can increase the number of training samples, thus improving its generalization ability and reducing the demand for a large number of samples.

Ref. [41] mentioned that there are two main approaches for implementing multi-task learning with parameter sharing: hard parameter sharing and soft parameter sharing.

In the hard parameter-sharing approach, the model architecture incorporates a shared hidden layer that is connected to task-specific output layers. The shared parameters facilitate the exchange of information and knowledge across tasks. Hard parameter sharing provides several advantages. Firstly, it effectively mitigates the risk of overfitting. By sharing parameters across tasks, the model is encouraged to learn generalized representations that capture common patterns and relationships among the tasks. This sharing of information prevents the model from becoming overly specialized for individual tasks, resulting in improved generalization performance on data. Secondly, hard parameter sharing can accelerate the training process, enabling the model to converge faster and achieve better performance across multiple tasks.

Soft parameter entails permitting each task to have its own models and parameters without necessitating exact similarity with parameters in other tasks. To encourage similarity among related parameters and diminish differences between shared layers, common constraints are introduced during training. These constraints impose regularization on the distance between parameters, encouraging them to become more similar and facilitating the sharing of information during training.

Ref. [42] employed an MTL approach based on deep belief networks (DBN) to predict short-term electricity, heat, and gas loads in integrated energy systems. Through information sharing between tasks, they observed improved prediction accuracy for all tasks. The electricity load forecasting task particularly benefited from the knowledge acquired from gas and heat profiles, resulting in enhanced predictions. Moreover, the MTL process proved to be more convenient and efficient than training each task independently.

Ref. [43] demonstrated the effectiveness of combining MTL with the least square support vector machine (LSSVM) algorithm for accurate load forecasting in integrated energy systems. The model improved load predictions and reduced training time compared to STL models. They also utilized grid search to identify optimal values for penalty coefficient and radial basis kernel function parameters.

Ref. [44] introduced a model with a crucial parameter “c”, and its impact on the model’s performance was investigated. Intriguingly, setting “c” to zero made the model function as an STL architecture solely focused on electricity load prediction. Ref. [44] highlighted the potential of MTL in improving the accuracy of electricity load forecasting. By incorporating the auxiliary task of temperature in the model, improved prediction performance was achieved, especially in commercial building scenarios.

A comprehensive analysis of the aforementioned studies reveals that training multiple related tasks concurrently in MTL offers advantages over independent task training. Consequently, compared to STL methods, MTL proves to be a more efficient approach, significantly reducing training time and substantially enhancing model accuracy.

Most of the existing research literature on prediction tasks has utilized single-task learning machine learning methods to study predictive models focused on a single learning objective. However, the limitations of single-task learning methods, such as underfitting and inadequate training with limited samples, may result in suboptimal model performance when dealing with complex problems. To address these issues, multi-task learning methods have been widely employed for complex predictive model problems. Additionally, multi-task learning allows the predictive model to simultaneously forecast multiple objectives, enabling it to obtain valuable information from other related auxiliary tasks, thereby reducing training time and improving the performance of each individual task.

In past studies that used multi-task learning for electricity load forecasting, electricity load prediction was the primary task, while temperature served as the auxiliary task. However, there has been limited exploration of cloud cover prediction as an auxiliary task. Considering the importance and impact of cloud cover and temperature on electricity load forecasting, this research focuses on electricity load forecasting for commercial buildings using a multi-task learning approach. The primary task is electricity load prediction, while temperature and cloud cover are employed as auxiliary tasks to construct the electricity load forecasting model for commercial buildings.

3. Research Methodology

This section will describe the research design, data collection and preprocessing, variable content, hyperparameter optimization, and model evaluation methods.

3.1. Research Scope and Design

As commercial buildings are recognized as one of the main contributors to energy consumption, significantly impacting climate change, businesses are increasingly formulating renewable energy procurement strategies. With ample rooftop space in commercial buildings for installing solar panels, which can operate under both sunny and cloudy weather conditions, solar energy can partially replace conventional electricity supply. This study aims to build a predictive model of commercial building electricity load using meteorological factors, such as environmental temperature, humidity, precipitation, and cloud cover. The research findings can serve as crucial references for enterprises to develop effective energy utilization strategies, ensuring a sufficient power supply throughout all periods and devising renewable energy procurement policies to reduce reliance on traditional electricity sources, thus aligning with ESG and sustainability initiatives.

The primary focus of this study lies in analyzing the electricity consumption data of a specific commercial building in Taiwan. Data will be collected from 1 January 2019 to 30 April 2021 at 15 min intervals. Simultaneously, climate data for the same period will be gathered to serve as feature variables for predicting electricity consumption.

The purpose of this study is to use actual electricity consumption data from a commercial building in Taipei and apply a multi-task learning approach to build a prediction model of electricity load. The primary task is electricity load forecasting, while temperature and cloud cover data are considered auxiliary tasks. The focus of the research is on employing deep neural networks (DNNs) to execute the multi-task learning (MTL) approach for electricity load, temperature, and cloud cover prediction. Additionally, some traditional single-task methods will be employed to construct the electricity load forecasting model, with the expectation that the multi-task learning approach will yield superior prediction results. The overall research design is illustrated in Figure 2.

3.2. Data Collection and Preprocessing

The process of data collection and data preprocessing stands as one of the most pivotal and challenging tasks within the research endeavor. This is predominantly due to the distinctive characteristics inherent to each dataset, which are intricately linked to specific objectives.

This study collected actual electricity consumption data spanning two consecutive years. The data collection period was extended from 1 January 2019 to 30 April 2021. The dataset encompassed electricity consumption data from a commercial building, as well as meteorological information. The entire dataset comprised a total of 20,424 individual data samples. The electricity consumption data was sourced from a commercial building in Taipei City, with electricity consumption readings recorded every 15 min and subsequently aggregated into hourly electricity consumption data. Additionally, this study obtained meteorological-related information from meteorological stations operated by the Central Weather Bureau of Taiwan. The meteorological information encompassed variables such as temperature, air pressure, relative humidity, wind speed, wind direction, precipitation, and cloud cover.

Following the acquisition of electricity consumption data from a commercial building in Taipei City and meteorological data from the Central Weather Bureau, data preprocessing procedures were undertaken to improve data quality and completeness. The data preprocessing tasks include data cleaning, anomaly detection, error data elimination, and missing value imputation. As an initial step, this study organized the electricity consumption data recorded every 15 min for the commercial building. Instances of electricity consumption data lower than 1000 were identified as anomalies and excluded from subsequent analyses.

3.2.1. Missing Data

The handling of missing data holds paramount importance in the data analysis process. Missing data can occur for various reasons, such as incomplete measurements, data entry errors, or participant non-responses. Failing to address missing data can lead to biased or inaccurate results, compromised statistical power, and reduced robustness of the findings.

The presence of missing data can introduce bias by influencing the representativeness of the sample and distorting the relationships between variables. Additionally, incomplete data can reduce the precision and reliability of statistical analyses, leading to erroneous conclusions. Therefore, proper handling of missing data is imperative to ensure the validity and credibility of research outcomes.

Imputing missing values through the utilization of machine learning models constitutes a widely adopted technique, often employing regression-based imputation methodologies. In this approach, existing variables are utilized as predictors to estimate and substitute missing values, as elucidated by [45]. This technique leverages historical data, enabling the machine learning model to grasp prevailing features and correlations while simultaneously considering intricate interdependencies to predict and substitute missing values, thereby circumventing the reliance on static imputation values.

However, instances arise in which the requisite historical data for features with missing values are unavailable. Employing machine learning models to impute these values may result in inadvertent biases. Such models might erroneously incorporate irrelevant features or misconstrue relationships, yielding imprecise imputations and thereby compromising predictive integrity.

Given the absence of historical data pertaining to cloud cover within this study, an alternative approach is embraced for imputation, addressing missing values within the cloud cover dataset. Consequently, the imputation methodology in this study resorts to the median for replacing missing values. This recourse not only obviates the dependence on historical data but also bases imputations on the intrinsic attributes of the dataset. This nuanced strategy ensures the employment of a more appropriate imputation technique.

3.2.2. Data Normalization

Data normalization plays a vital role in the initial stages of data preprocessing, aiming to standardize and rescale the features within a dataset. The primary purpose of data normalization is to keep the model training process from being dominated by features with higher magnitudes. In this article, min–max normalization is employed to transform the feature by converting the numerical values to the range between the minimum value of zero and the maximum value of one. This transformation is accomplished using the formula presented in Equation (1) as described below.

(1)$X_s={\displaystyle \frac{x-min\left(x\right)}{max\left(x\right)-min\left(x\right)}}$

In this equation, the variables represent different aspects. The original value of the feature is denoted by $x$, while min(x) and max(x) refer to the minimum and maximum values within the vector $x$. The value $X_s$ represents the data to be normalized. The utilization of min–max normalization will enable the rescaling of meteorological data within the article to a common range. This normalization process facilitates more effective and balanced model training.

3.2.3. Data Splitting

In the context of developing predictive models, the process of data splitting into training, testing, and validation datasets plays a critical role in assessing and optimizing model performance. This procedure ensures that the model’s generalization capability is properly evaluated and prevents overfitting. The dataset is typically divided into three distinct subsets: the training set, which the model learns from, the testing set, which evaluates the model’s performance on unseen data, and the validation set, which assists in tuning hyperparameters. The training set constitutes the largest portion and serves as the foundation for model training. The model learns patterns, relationships, and features within this subset, allowing it to formulate predictions. The testing set remains separate and is never exposed to the model during training. It evaluates the model’s predictive accuracy on new, unseen data. This assessment helps gauge the model’s generalization capability and potential overfitting, which occurs when a model performs exceptionally well on training data but poorly on unseen data.

In this study, the data were divided into three subsets using a ratio of 60%, 20%, and 20%, respectively. As shown in Figure 3, these subsets were defined as the training data, testing data, and validation data. Specifically, 60% of the data was allocated for constructing the training predictive model. Subsequently, 20% of the data was employed for hyperparameter tuning, validating the predictive model, and ultimately evaluating model performance. The final 20% of the data was dedicated to testing the predictive model and assessing its generalization capability.

3.3. Research Framework

Ref. [44] proposed a multi-task learning model to predict the hourly electricity load in commercial buildings. The main task of the model was hourly electricity load, and the temperature prediction was considered an auxiliary task. Continuing from the results of preceding research and drawing from a comprehensive analysis of the literature, it has been discerned that cloud cover exhibits a notable and statistically meaningful correlation with electricity consumption. Moreover, it has been observed that there exists an inherent interdependence among electricity consumption, temperature, and cloud cover. Building upon these empirical observations, this study aims to extend the scope of inquiry by integrating cloud cover prediction as an auxiliary task within a multi-task learning framework, thus facilitating the formulation of a sophisticated electricity consumption prediction model. The primary objective of this investigation is to develop an innovative electricity consumption prediction model characterized by a multi-task learning approach. The major task is electricity consumption forecasting, while the auxiliary tasks are the prediction of temperature and cloud cover. The research framework, which is illustrative of this multifaceted approach, is depicted in Figure 4.

The model architecture integrates fully connected layers, also known as linear layers, within both the shared hidden layers and task-specific output layers. These layers are engaged subsequent to the input of features into the model. The initial linear layer’s input dimension is determined by the specified input size. The output of each layer is sequentially directed as input to the ensuing layer, thus ensuring a continuous flow of information within the network. This systematic progression ensures a gradual transformation and processing of data, allowing the network to derive meaningful insights from input data and learn hierarchical representations. Through this cascading process, the model progressively captures intricate patterns and relationships in the data.

Furthermore, to capture the non-linear characteristics intrinsic to the data and heighten the model’s predictive capacity, our study incorporates batch normalization (BN) [46] and rectified linear unit (ReLU) functions [47]. These functions are applied across both the shared hidden layers and task-specific layers. The incorporation of BN adjusts the data distribution, enhancing training stability and expediting convergence. Concurrently, the ReLU function introduces nonlinearity, empowering the model to effectively capture intricate relationships embedded within the data.

The shared hidden layers encompass a total of five strata, commencing with a 512-dimensional feature vector as input, sequentially followed by layers of 512, 256, 128, 64, and 32 neurons. Similarly, the main task output layer and the two auxiliary task output layers feature a dual-tier structure, with the first hidden layer consisting of 32 neurons and the second hidden layer housing 16 neurons. These layers adhere to a uniform configuration, inclusive of linear layers, BN, ReLU, and a dropout rate of 0.05, which is systematically applied across each stratum. This consistent framework guarantees parity in normalization, activation, and regularization techniques across the task-specific layers of the network.

In essence, the proposed model employs a hybrid approach, merging a shared network with task-specific networks to undertake MTL. Given the regression nature of all forecasting tasks, the model benefits from augmented data normalization, non-linear mapping, and regularization via the incorporation of BN, ReLU, and dropout techniques. This amalgamation facilitates the adept capture of intricate data patterns and representations, simultaneously mitigating the potential pitfalls of overfitting attributed to an excessive parameter count. The specifics of the envisaged model architecture will be delineated in the subsequent sections.

3.3.1. Batch Normalization

Batch normalization is a commonly utilized technique for feature normalization in neural networks. Since the distribution of the data in each layer is not the same, the parameters will be adjusted as each layer is trained, and the distribution of the input data will change significantly with each adjustment.

Hence, during the training process, the information of each layer will be standardized to avoid internal covariate shifts so that the output conforms to a normal distribution with a mean of zero and a standard deviation of one, making the data distribution more even. The advantage of batch normalization is to help converge to the optimal value more quickly.

Furthermore, γ and β are the learnable parameters used to scale and shift the normalized values, respectively. The batch normalization is formulated in Equation (2).

(2)$X_{BN}={\displaystyle \frac{x-E\left(x\right)}{\sqrt{Var\left(x\right)+\epsilon }}}\times \gamma +\beta$

where $X_{BN}$ is the new value after normalization, $E\left(x\right)$ is the mean within a batch, $Var\left(x\right)$ is the variance within a batch, and $\epsilon$ is an arbitrarily small constant for numerical stability.

3.3.2. Activation Function

In the realm of deep learning, should activation functions be excluded from a neural network, the resultant relationship between input and output would remain linear. The activation function serves as a non-linear equation that introduces nonlinearity into the neural network. The selection of distinct activation functions can be tailored to suit specific problem requisites. Given the prevalence of nonlinearity in many real-world scenarios, the incorporation of non-linear activation functions becomes imperative. Failing to incorporate these non-linear activation functions would render the model’s interpretation or effectiveness devoid of significance. The prevailing types of activation functions are succinctly expounded upon as follows [48]. ReLU has gained significant popularity as an activation function in recent years.

On the contrary, if the output value is positive, which means it is greater than zero or equal to zero, it keeps the original value unchanged and enters directly. This property allows for simplified computational complexity, faster calculations, and accelerated convergence of the model, as only the condition of being greater than zero needs to be checked during output. The derivative of the ReLU function remains a constant value of 1 when the input is greater than zero. This characteristic overcomes the issue of vanishing gradients.

Nevertheless, the ReLU function encounters a significant limitation known as the dead ReLU problem. When the input is negative, the output of the ReLU function becomes 0, resulting in a derivative of 0, as well.

Consequently, the neurons become inactive, impede effective weight updates during the backpropagation process, and struggle to learn effectively. It is crucial to set the appropriate learning rate to mitigate this problem when utilizing the ReLU function.

3.3.3. Dropout

The dropout technique is an essential regularization method in the realm of deep learning employed to mitigate the issue of overfitting. During the model training process, dropout is typically applied by stochastically setting a portion of neurons’ outputs to zero. This practice enables the model to avoid heavy reliance on specific neurons, thereby enhancing its generalization capability. This stochastic dropout of neurons can be likened to training multiple independent neural networks, effectively reducing the likelihood of overfitting.

The significance of the dropout technique lies in its contribution to enhancing model robustness and generalization ability. By randomly deactivating portions of neurons, the model is compelled to learn different combinations of features. This aids in diminishing co-adaptation among neurons, preventing the model from overly fixating on the noise present in the training data. This is crucial for handling intricate real-world data and ensuring commendable performance on unseen data instances. Consequently, the application of the dropout technique in deep learning is widespread, ensuring stable and accurate model performance across various applications.

Ref. [49] research introduced the dropout technique, as depicted in Figure 5. Through the random deactivation of distinct neurons, the model diminishes its dependence on specific features, effectively countering overfitting problems. Dropout, when integrated into the model construction process, serves as a form of regularization, enabling neural networks to learn in a more robust manner.

3.3.4. Optimizer

The optimization process in training holds a pivotal role, as it encompasses the responsibility of fine-tuning the learning rate for parameter optimization and the achievement of optimal parameter values. By employing the gradients calculated during the backpropagation step, the optimizer dictates the strategy for updating the model’s parameters.

Several widely utilized optimizers are present in deep learning, including stochastic gradient descent (SGD), root mean square propagation (RMSprop), and adaptive moment estimation (Adam). Each optimizer possesses distinct strengths and may exhibit varying performance based on the specific context.

In a study conducted by [50], it was evidenced that adaptive moment estimation (Adam) offers significant advantages due to its proficient parameter updates, enhanced stability, reduced memory requirements, and adaptive learning rate capabilities in practical scenarios.

A prominent advantage of Adam lies in its efficiency when updating model parameters. By integrating the traits of adaptive learning rate and momentum, it achieves more efficient and stable parameter updates. This efficiency proves particularly valuable when dealing with extensive datasets or a substantial number of parameters. Furthermore, Adam contributes to lowered memory usage. By leveraging adaptive learning rate adjustment capabilities, it optimizes memory consumption during the optimization process. This characteristic renders Adam well-suited for contexts with limited memory resources. Another noteworthy attribute of Adam is its adaptive learning rate feature. The learning rate is dynamically adjusted based on insights gained from past gradient updates of the parameters. This adaptability equips Adam to adeptly handle scenarios with evolving dynamics, facilitating convergence to optimal solutions at various stages of the training procedure. In view of these merits, Adam is selected as the preferred optimizer for this article.

3.3.5. Learning Rate

The learning rate is a critical hyperparameter in deep learning that is employed to regulate the magnitude of model parameter updates in each iteration. The selection of the learning rate significantly impacts the training and convergence process of the model. The learning rate determines the step size at which the model moves during each parameter update. If the learning rate is set too small, the convergence speed of the model becomes sluggish, necessitating more iterations to reach convergence. Conversely, if the learning rate is set too large, it may cause the model to skip over the optimal solution during the update process or even prevent convergence, resulting in an unstable loss function.

Selecting an appropriate learning rate is a delicate balancing act. Generally, adjusting the learning rate’s magnitude can expedite convergence; however, it is crucial to ensure that the model converges stably to the optimal solution. In the initial stages of training, a larger learning rate can aid the model in rapid learning, but as the training progresses, the learning rate often needs to be gradually reduced to avoid oscillations or overshooting near the optimal solution.

Different optimization algorithms and model architectures might require varying optimal learning rates. Typically, experimenting with different learning rate values while monitoring the model’s training progress and observing changes in the loss function and improvements in accuracy are essential. Some optimization algorithms, such as stochastic gradient descent (SGD) and adaptive moment estimation (Adam), could possess adaptive learning rate capabilities, allowing for different learning rates to be utilized during different training stages.

The learning rate is of paramount importance in deep learning, as it directly influences the training effectiveness and convergence speed of the model. Precisely adjusting the learning rate can facilitate more effective learning and better performance. In this study, the effectiveness of dynamically adjusting the learning rate based on the validation dataset’s results was validated. By gradually decreasing the learning rate, the model tends to stabilize and converge. Therefore, progressively reducing the learning rate throughout the entire model training process enhances the model’s convergence to superior solutions and improves overall performance. Figure 6 presents the learning curves of our proposed model, where the y-axis represents the loss value, and the x-axis indicates the number of epochs. The results show that adjusting the learning rate helps reduce the validation loss as the number of epochs increases.

3.4. Hyperparameter Optimization

Hyperparameter optimization constitutes a pivotal facet within the realm of machine learning and deep learning workflows. This intricate process revolves around the quest for an optimal assemblage of hyperparameters tailored to a given machine learning model, algorithm, or neural network architecture. Hyperparameters, which are distinct from parameters directly learned through data, wield a considerable influence over a model’s behavior, performance, and eventual efficacy. The judicious calibration of hyperparameters exerts a profound impact on facets such as model performance, convergence dynamics, and the model’s ability to generalize beyond training data. Hyperparameter optimization assumes a central role in the pursuit of optimal model performance. This multifaceted endeavor encompasses an array of methodologies and techniques that systematically probe and unravel the optimal amalgamation of hyperparameters, underscoring their indispensable stature within the triumphant landscape of machine learning and deep learning ventures. Prudent hyperparameter tuning notably heightens model precision, convergence tempo, and generalization potential, positioning them as integral constituents of prosperous machine learning undertakings.

The present study employs the Optuna methodology for the purpose of identifying optimal parameter configurations within multi-task learning (MTL) models. Furthermore, the random search technique is adopted to ascertain the optimal parameter combinations within single-task learning (STL) models.

Optuna, as introduced by [51], is characterized by its flexible integration capabilities with prevalent algorithms. This methodology employs a dynamic search strategy, wherein parameter adjustments are informed by trial outcomes. This adaptive approach contributes to the expedited convergence of trials, ultimately leading to the efficient identification of optimal combinations within a reduced timeframe. Conversely, the random search approach, posited by [52], adopts a distinctive methodology. It uniformly samples hyperparameter combinations from a predefined range, facilitating the selection of the most efficacious set of hyperparameters.

In synopsis, the present article harnesses the prowess of both the Optuna and random search methodologies. By incorporating these methods, the research endeavors to efficiently explore optimal hyperparameter combinations. This dual-pronged approach enhances the efficacy of the search process, thereby elevating the effectiveness of parameter optimization endeavors. This paper uses the categorical function with specified ranges of the hyperparameters of Optuna to perform the hyperparameter optimization process.

3.5. Loss Functions

The training process of a model is intricately influenced by both the design of its architecture and the careful selection of loss functions and weight adjustments. Serving as a metric to gauge the dissimilarity between predicted and actual values, the loss function assumes the role of an objective function during the model’s training phase.

A reduced loss function value signifies a heightened alignment between predicted and actual values, thus indicating improved model performance. During the backpropagation process, the comprehensive error computed by the loss function exerts a direct impact on parameter updates, dictating both the direction and magnitude of these updates. Hence, the judicious selection of an appropriate loss function and the meticulous adjustment of task weights wield a palpable influence on the model’s learning trajectory and overall performance.

Within the scope of this article, in which both primary and auxiliary tasks encompass regression problems, the ${\mathcal{L}}_1$ loss function, commonly referred to as the mean absolute error (MAE), is deemed the fitting choice. The proposed model introduces dedicated loss functions for both the primary task and auxiliary tasks, which are elegantly formalized in Equations (3)–(5). ${\mathcal{L}}_{main}$ characterizes the loss function pertaining to electricity load forecasting, while ${\mathcal{L}}_{aux1}$ and ${\mathcal{L}}_{aux2}$ respectively represent the loss functions for the first auxiliary task (cloud cover) and the second auxiliary task (temperature), wherein $y_i^e$, $y_i^c$, $y_i^t$ are the observed values for electricity load, cloud cover, and temperature, and ${\widehat{y}}_i^e,$ ${\widehat{y}}_i^c$, ${\widehat{y}}_i^t$ are the predicted values for electricity load, cloud cover, and temperature.

This strategic approach remarkably enables the concurrent management of diverse tasks, each endowed with distinct degrees of significance. The primary task, which is centered around electricity load forecasting, is thoughtfully demarcated from the auxiliary tasks, involving forecasts of temperature and cloud cover.

(3)${\mathcal{L}}_{main}=\sum _{i=1}^n\left|y_i^e-{\widehat{y}}_i^e\right|$

(4)${\mathcal{L}}_{aux1}=\sum _{i=1}^n\left|y_i^c-{\widehat{y}}_i^c\right|$

(5)${\mathcal{L}}_{aux2}=\sum _{i=1}^n\left|y_i^t-{\widehat{y}}_i^t\right|$

A marked discrepancy in data scales is evident between the primary task and auxiliary tasks, wherein electricity load values exhibit pronounced magnitudes in contrast to the others. Addressing this concern, an introduced weight parameter (denoted as ω) undertakes the role of recalibrating the relative importance attributed to the main and auxiliary tasks within the loss function framework. Precisely, the auxiliary task loss undergoes multiplication by this weight parameter, as delineated in Equation (6).

(6)$\mathcal{L}={\mathcal{L}}_{main}+{\omega }_1{\mathcal{L}}_{aux1}+{\omega }_2{\mathcal{L}}_{aux2}$

The weights ${\omega }_1$ and ${\omega }_2$ are assigned to the distinct auxiliary tasks, namely, cloud cover and temperature, encapsulating their distinct significance within the holistic loss function paradigm. Through the discrete computation of losses for the main task and auxiliary tasks, followed by their amalgamation with the pertinent weights, the underlying aim is to achieve an equilibrium in task importance throughout the model training process.

3.6. Evaluation Metrics

In the realm of evaluating predictive models, mean absolute error (MAE) and root mean square error (RMSE) hold paramount importance as robust metrics for quantifying the accuracy and reliability of predictions against actual values. MAE and RMSE are pivotal tools for assessing model accuracy, enabling the meaningful interpretation of prediction errors, and aiding in the selection and optimization of predictive models. Their distinct characteristics and suitability for varied scenarios make them indispensable components of the model evaluation toolkit.

MAE is defined as the average of the absolute differences between predicted and actual values. MAE provides a straightforward measure of prediction accuracy, considering both overestimations and underestimations. It yields a clear representation of the magnitude of errors, expressed in the original unit of the target variable. MAE’s ability to be directly interpretable makes it particularly suitable for scenarios in which the understanding of prediction errors in real-world units is vital.

The RMSE is derived by taking the square root of the average of the squared differences between predicted and actual values. The RMSE offers a more pronounced emphasis on larger errors due to the squaring operation. It effectively penalizes larger deviations between predictions and actual values, making it sensitive to outliers. The RMSE tends to reflect the dispersion of errors and provides a measure of precision, albeit in a squared context. Its robustness against outliers and capacity to quantify prediction variability make it valuable for tasks in which reducing significant errors is a priority.

The complex task of electricity load forecasting is meticulously tackled within the purview of a regression framework. As a natural consequence of this approach, the evaluation of the MTL deep learning model’s efficacy is underpinned by the adoption of two intrinsic appraisal metrics, namely, MAE and RMSE.

These evaluative benchmarks serve as quantitative yardsticks, methodically employed to measure the degree of dissimilarity between the predicted outcomes and the actual observations. By encapsulating these key aspects, the assessment endeavor achieves a comprehensive elucidation of the model’s precision in the domain of electricity load forecasting. The formulations for MAE and RMSE are elegantly articulated in Equations (7) and (8), respectively.

Within these equations, the symbol $\widehat{y_i}$ signifies the predicted value, $y_i$ denotes the actual observed value, and n embodies the aggregate count of data samples. This principled approach to performance evaluation provides a sound foundation for the quantitative scrutiny and interpretation of the MTL deep learning model’s proficiency in addressing the intricate challenges of electricity load forecasting.

(7)$MAE= {\displaystyle \frac{1}{n}}\sum _{i=1}^n\left|\widehat{y_i}-y_i\right|$

(8)$RMSE=\sqrt{{\displaystyle \frac{1}{n}}\sum _{i=1}^n{\left(\widehat{y_i}-y_i\right)}^2}$

4. Research Findings and Analysis

In this study, the methodology of multi-task learning is employed, with the primary objective being the prediction of electricity load. Simultaneously, auxiliary tasks encompassing temperature and cloud cover are incorporated. This innovative research design enables the concurrent prediction of all three tasks. Through the utilization of multi-task learning, shared features are harnessed to enhance prediction performance and glean insights into the interrelationships among these variables.

Furthermore, a comparative analysis is undertaken between the multi-task learning approach and several traditional single-task learning methods to assess the efficacy of the proposed multi-task learning paradigm. This investigation serves as a comprehensive evaluation of the predictive capabilities of the proposed approach.

This section will expound upon the experimental design, the comparative methodologies employed, the results pertaining to hyperparameters, and ultimately, the empirical outcomes.

4.1. Experimental Settings

This study utilizes actual electricity load data from a commercial building in Taiwan to investigate the correlation between electricity load and meteorological information, ultimately constructing a predictive model for electricity load. The data collection period spans from 1 January 2019 to 30 April 2021, encompassing an initial dataset of 20,424 records. Following a comprehensive data preprocessing procedure, which involves the removal of missing data and instances unsuitable for the predictive model establishment, a refined dataset of 20,090 records is obtained.

Ref. [53] has underscored a significant correlation between time variables and short-term electricity demand forecasting for office buildings. In light of this, the input features encompass a diverse array of meteorological factors. Furthermore, the incorporation of calendar-related data, including weekday and weekend indicators, along with seasonal indexes, forms an integral part of the input features. Based on the scope of this article, the MTL model is trained with the ReLU activation function applied to each layer. The MAE serves as the designated loss function, while the Adam optimizer is employed. To mitigate overfitting, a dropout rate of 0.05 is implemented.

The concept of epochs, constituting a complete iteration of the model’s training process with the entire dataset, is explored. The appropriate number of epochs is contingent on task characteristics and dataset attributes, with no fixed or absolute value.

Batch size, which determines the number of training samples per batch, is a pivotal factor in balancing efficiency and effectiveness. In this study, the training process involves setting the epoch value at 300, and experimentation with varying batch sizes (100, 128, 150, and 200) is conducted. The rationale for choosing these batch sizes is to treat batch size as a hyperparameter to be tuned along with the learning rate and use heuristic methods to conduct experiments to find the most suitable batch size. Through this exploration, the optimal batch size that maximizes model effectiveness is identified.

Hyperparameters are explored using the Optuna method in a randomized fashion. The resulting optimal hyperparameter combination is then applied to the MTL model, which simultaneously addresses electricity load, temperature, and cloud cover prediction tasks. An iterative procedure is executed, yielding 10 sets of results that are subsequently averaged to form a robust ensemble prediction. This ensemble approach enhances the precision and reliability of integrated predictions, thereby bolstering the overall stability of the forecasting process.

Concurrently, conventional STL methods are employed as a comparative benchmark, with their hyperparameters explored through the random search method. This facilitates a rigorous comparison between the predictive performance of the MTL model and traditional methods in electricity load forecasting.

Model performance evaluation hinges on the MAE and root mean square error (RMSE) metrics. The MTL model, which is optimized with the determined batch size and hyperparameter combination, undergoes an in-depth analysis and comparison against the outcomes of conventional STL methods. This meticulous scrutiny affords insights into the merits and potential value of the MTL model within the realm of electricity load forecasting.

4.2. Comparison Methods

To assess the efficacy of the MTL model proposed in this article, by effectively mastering multiple interconnected tasks and demonstrating superiority over conventional STL models, a comprehensive comparative analysis is undertaken. This analysis specifically involves a range of conventional STL models, each tailored and evaluated for the distinct task of electricity load forecasting.

Within the context of this article, representative STL models encompass decision tree (DT), random forest (RF), eXtreme gradient boosting (XGBoost), and long short-term memory (LSTM). This selection aims to provide a holistic comprehension of the strengths and practicality inherent to the MTL model.

A decision tree (DT) is a non-parametric method characterized by flexibility and the capacity to handle diverse data types. It sequentially partitions datasets based on varying conditions or features until the information gain either converges to zero or falls beneath a specified threshold [54]. This approach’s simplicity and interpretability render it widely applicable across various domains and applications.

The random forest (RF) model, as an ensemble learning technique, integrates multiple regression decision trees to forecast target outcomes. Each individual regression decision tree is independently generated while collectively maintaining a consistent distribution across the entire random forest [55].

Leveraging parallel processing capabilities, eXtreme gradient boosting (XGBoost), which is a gradient boosting tree model, adeptly manages training and prediction tasks for multiple sub-models simultaneously. This feature contributes to accelerated training speed and overall model performance [56].

In the realm of recurrent neural networks (RNNs), long short-term memory (LSTM) stands as a notable advancement. Its prowess is particularly manifest in time series forecasting tasks, having demonstrated successful outcomes in electricity load forecasting endeavors [57]. The inherent competence of LSTM in handling temporal dependencies positions it as an effective tool for capturing and predicting time series patterns in electricity consumption.

Having introduced the selected representative STL methodologies, the subsequent segment of this article delves into the outcomes of parameter tuning for the proposed MTL model. Concurrently, the parameter tuning results for the STL models are presented, culminating in a comprehensive comparison to illuminate the comparative advantages and potential efficacy of the MTL model in electricity load forecasting.

4.3. Hyperparameter Results

By conducting parameter tuning experiments, the MTL model’s performance can be comprehensively evaluated across various hyperparameter configurations. This facilitates a rigorous comparative analysis with conventional STL models, shedding light on the MTL model’s effectiveness and adaptability under diverse conditions. These insights contribute to identifying the optimal parameter configuration for achieving peak performance.

In the initial section, this article presents the outcomes of hyperparameter configuration specific to the MTL models. This exploration includes different batch sizes—100, 128, 150, and 200—whose results are methodically compiled in Table 1. It is important to emphasize that consistency is maintained across all MTL experiments through the uniform application of a fixed number of epochs, set at 300.

During model training, an initial warm-up phase spanning 35 epochs is executed. Throughout this preliminary phase, the model is trained using an initial learning rate of 0.11. Upon the conclusion of the warm-up phase, an early-stopping mechanism is triggered. If the validation loss fails to exhibit improvement over 45 consecutive epochs, training is prematurely terminated. In such scenarios, the learning rate is dynamically adjusted to 0.0002, in tandem with the concurrent adaptation of model parameters. This adaptive learning rate modulation, which is based on the validation set’s performance, optimizes the model’s trajectory throughout the training process.

In our proposed loss functions, we use hyperparameters ${\omega }_1$ and ${\omega }_2$ to control the weights of the two auxiliary tasks (${\omega }_1$ for cloud cover and ${\omega }_2$ for temperature), and we propose to use the ratio of the average electricity load and the average temperature and average cloud cover to determine the values of ${\omega }_1$ and ${\omega }_2$, respectively, which are 180 and 85 in this work. These weight allocations were chosen strategically to enhance the precision of the primary electricity load task.

Furthermore, this manuscript systematically outlines the outcomes of hyperparameter tuning for representative STL models, which is conveniently summarized in Table 2 and Table 3. These tables collectively offer a comprehensive snapshot of the diverse models under consideration, effectively illustrating the distinct parameter configurations adopted for both MTL and STL models.

4.4. Experimental Results

The experimental findings and the subsequent model performance comparison are delineated, leveraging the specified evaluation metrics. By undertaking a comprehensive scrutiny and assessment of the multi-task model while juxtaposing its outcomes with those of single-task methodologies, an enhanced comprehension of MTL learning’s potential, merits, and boundaries for real-world applications is attainable.

By the outcomes elucidated in Table 4, it becomes evident that the distinct MTL models—identified as MTL-1 through MTL-4, each associated with batch sizes of 100, 128, 150, and 200, respectively—outperform the four STL models about both MAE and RMSE metrics. The units of MAE and RMSE are Kilowatt-hours (kwh), corresponding to the original unit of electricity load data.

The MTL models consistently yield superior outcomes, which are characterized by the lowest MAE and RMSE values, irrespective of the chosen batch size. This pattern underscores the efficacy of the proposed MTL methodology in accurately forecasting the electricity load while maintaining relatively minor discrepancies when compared to the actual observed values.

Furthermore, it becomes conspicuous that MTL-3, configured with a batch size of 150, attains the most remarkable performance among all MTL models, with an MAE of 227.99 and an RMSE of 346.53. The experimental results indicate that our proposed method significantly outperforms the other compared methods on traditional single-task methods. Moreover, we plot the prediction outcomes of the proposed models by employing a batch size of 150 (MTL-3) and the observed loads in a day. Figure 7 shows the results. The differences between the predicted loads of the proposed model and the observed loads are small, which is why the proposed model can provide the best performance. This also substantiates the critical role of appropriate batch size in attaining the pinnacle of predictive prowess for the MTL model.

In summation, the MTL approach enriched with auxiliary tasks demonstrates a heightened efficacy in elevating the performance of the principal task, in contrast to the STL methodology, which lacks auxiliary tasks.

This study exhaustively delves into the realm of MTL deep learning in the domain of electricity load forecasting. The outcomes unveil the triumphant creation of a deep model-driven forecasting framework that is adeptly capable of precise electricity load prediction across multiple interrelated tasks. The forthcoming section will accentuate the research contributions inherent in this study and proffer suggestions for prospective avenues of investigation.

5. Conclusions, Contributions, and Suggestions

5.1. Conclusions

This study employed a multi-task deep learning model to undertake electricity load forecasting and analysis within a commercial building context. The main focus of the model was to predict electricity load, with supplementary tasks encompassing temperature and cloud cover forecasting. This study delved into diverse parameter configurations, examining their implications on model performance and drawing comparisons with various representative single-task learning (STL) approaches.

Throughout the model training phase, an array of parameter settings was explored and scrutinized across distinct batch sizes—100, 128, 150, and 200—and their effects on model performance were meticulously observed. The outcomes unveiled that the model achieved optimal performance when utilizing a batch size of 150, yielding the lowest values for MAE and RMSE. Consequently, the judicious selection of an appropriate batch size emerged as a pivotal determinant in augmenting the model’s forecasting precision and generalization capabilities. Additionally, this investigation ventured into diverse weight assignments for the two auxiliary tasks. The results underscored the effectiveness of allocating a weight of 180 to cloud cover and 85 to temperature, leading to improved electricity load forecasting outcomes.

The rationale behind this weight distribution emanates from the recognition that fluctuations in cloud cover can impact solar power generation efficiency, influence indoor lighting conditions, and subsequently influence lighting equipment demand. Consequently, heightened cloud cover might necessitate additional lighting fixtures to counter diminished natural light, consequently escalating electricity demand. Conversely, diminished cloud cover and increased natural light could reduce electricity consumption.

Conversely, while temperature received a comparably lower weight, it by no means suggests an absence of correlation with electricity consumption. Temperature maintains its relevance in relation to electricity consumption, albeit likely to a lesser extent than cloud cover. Temperature fluctuations predominantly influence cooling requirements within buildings, subsequently affecting the operation of air conditioning systems. With rising temperatures, the demand for air conditioning surges in order to enhance comfort and lower indoor temperatures, thus elevating electricity consumption. Although temperature’s assigned weight is relatively modest, its correlation with electricity load forecasting remains evident, albeit with intricate influencing factors.

In closing, this research deduces that within the employed MTL model, cloud cover assumes a pivotal role, while temperature, although relatively less prominent, remains germane. These findings bear substantial implications for energy management practices and electricity load forecasting within commercial building contexts.

5.2. Contributions

The inception of this research delineated the research objectives, with the principal aim being the implementation of a multi-task deep learning model for electricity load forecasting within a commercial building context. The focal point of this endeavor revolves around the anticipation of electricity load, while auxiliary tasks encompass the prediction of temperature and cloud cover. Guided by this overarching objective, the article bestows the following contributions upon the academic sphere:

It integrates auxiliary tasks, specifically temperature and cloud cover, into the electricity load forecasting paradigm via the utilization of multi-task deep learning techniques. This amalgamation serves to amplify the precision and dependability of the forecasting model, leading to a pronounced enhancement in the overarching predictive performance.

It provides a distinct enhancement in practicality, and applicability is realized by harnessing real-world data for analysis. Consequently, the insights and pragmatic strategies offered in this article hold pertinent implications for energy management practices within commercial edifices, catering to real-world exigencies.

The pivotal role played by accurate electricity load forecasting as a bedrock for power system planning and grid management cannot be overstated. This facet contributes to cost-saving endeavors and improved operational milieu for enterprises while simultaneously furnishing a foundation for informed decision-making, including the formulation of strategies for sustainable and ecologically viable energy procurement.

The imperative of precise electricity load forecasting reverberates in power system planning and grid management, facilitating efficient energy resource allocation, thereby resulting in financial economies and an ameliorated operational environment. Simultaneously, it provides a conduit for sagacious decisions encompassing sustainable energy procurement schemes and judicious energy utilization strategies.

The latent potential of this research augments the cause of enhanced energy efficiency and aligns harmoniously with the pursuits delineated within the United Nations Sustainable Development Goals (SDGs). These encompass a broad spectrum, SDG-7, which underscores the expansion of renewable energy integration, SDG-13, which encapsulates the thrust toward climate action goals, and an overarching momentum propelling progress toward the tenets of sustainable development.

5.3. Suggestions for Future Research

This study employs the electricity consumption data from a commercial building located in Taipei City, Taiwan, for research and analysis. It is worth noting that the results obtained might not be fully representative of different regions, various types of commercial structures, or distinct energy demand patterns and power load characteristics in other settings, such as residential and medical centers. The peak electricity consumption for commercial buildings typically occurs during daytime office hours, while for residential buildings, it might peak in the evening after work hours. Electrical equipment in commercial buildings is primarily composed of lighting fixtures, heating, ventilation, and air conditioning systems, as well as general household appliances. Conversely, hospitals and medical centers require an uninterrupted 24/7 power supply to ensure unhindered medical operations. Consequently, it is imperative to recognize that future research could encompass diverse geographical regions, different building types, and their respective energy consumption characteristics. The development of individualized electricity consumption prediction models for each type of structure can facilitate a comprehensive comparison of various building-specific prediction models, culminating in a unified electricity consumption prediction model applicable across all building types. Such a model would serve as a pivotal reference for formulating electricity procurement strategies within the industry and could also inspire diverse research methodologies and designs in academia.

Additionally, the cloud cover data utilized in this study, which were derived from manual records during office hours (8:00–17:00), are restricted to a specific time frame; thus, data from other periods are lacking. Moreover, due to subjective human judgment, inconsistencies in standards and data errors could arise, challenging the accuracy and completeness of cloud cover data. Consequently, preprocessing and data imputation are prerequisites for the cloud cover data before establishing prediction models. Nevertheless, these data-handling processes may introduce biases and potentially impact the accuracy of prediction models. Future studies could collaborate with meteorological experts to collect more comprehensive cloud cover data, including the amalgamation of observations from multiple stations or the incorporation of diverse meteorological resources to procure broader meteorological datasets. Such an approach would facilitate the exploration of advanced techniques for cloud cover data imputation, thereby elevating the precision of cloud cover data.

Concerning the design of the prediction models, this study predicts the next hour’s electricity consumption using all available feature variables. It is imperative to acknowledge that the electricity load of commercial buildings might exhibit different variation patterns within distinct time ranges. Hence, future research could extend its scope to encompass various time spans and intervals, dynamically incorporating temporal aspects into the model. By establishing models that span diverse time ranges, a more diverse range of experimental outcomes can be attained, leading to an enhanced ability to capture the variability in electricity load.

In considering auxiliary tasks, this research confirms that the inclusion of auxiliary tasks, such as temperature and cloud cover prediction, enhances the accuracy of electricity consumption prediction models, thereby augmenting the efficacy of multi-task learning (MTL) models. However, it is important to recognize that other factors, such as humidity and precipitation, could also impact electricity consumption. Future studies could incorporate these factors as auxiliary tasks within the MTL model to improve the precision and reliability of electricity consumption predictions. Consequently, it is recommended that future work focuses on encompassing a wider array of relevant auxiliary tasks, thereby better capturing the diverse influences on electricity consumption and enhancing the overall predictive capabilities of the model.

In summary, this study employs a multi-task deep learning approach, integrating two auxiliary tasks—temperature and cloud cover—to enhance the accuracy and reliability of electricity consumption prediction models. The research findings provide robust support for energy management within commercial buildings. Future studies could extend their scope to include a broader spectrum of distinctive research subjects and incorporate additional relevant auxiliary tasks. This expansion could furnish more comprehensive information and refined prediction results for energy planning and management. Consequently, this not only broadens the horizons of energy planning and management but also facilitates the formulation of energy-efficient and effective management strategies, ultimately boosting energy efficiency.

Author Contributions

Conceptualization, K.-B.H. and T.-S.L.; methodology, H.-M.L.; software, J.L. and J.-P.W.; validation, K.-B.H., H.-M.L. and J.L.; formal analysis J.-P.W.; investigation, J.-P.W. and J.L.; resources, L.L. and K.-B.H.; data curation, H.-M.L.; writing—original draft preparation, J.-P.W. and J.L.; writing—review and editing, L.L. and T.-S.L.; visualization, H.-M.L.; supervision, K.-B.H. and T.-S.L.; project administration, H.-M.L.; funding acquisition, K.-B.H. and L.L. All authors have read and agreed to the published version of the manuscript.

Data Availability Statement

Data is unavailable due to commercial privacy.

Acknowledgments

The authors would like to thank the editor and anonymous reviewers for their valuable and constructive comments, which have led to the significant improvement of this study. Specially thanks to the National Science and Technology Council in Taiwan and all the teams in Fu Jen Catholic University for all administration supports.

Conflicts of Interest

The authors declare that there are no conflicts of interest pertaining to this research study. We affirm that this submission is free from any financial, personal, or professional interests that could influence the objectivity, integrity, or credibility of the research outcomes presented in the manuscript. We disclose that we have no financial relationships, affiliations, or competing interests that could be perceived as exerting an undue influence on the interpretation of the findings or conclusions of this study. This includes but is not limited to financial holdings, grants, honoraria, consultancies, employment, directorships, patents, or other relationships with entities that could be construed as having a vested interest in the results of this research. We hereby confirm that this declaration accurately represents the absence of any conflict of interest in relation to the submitted manuscript.

Footnotes

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Figures and Tables

Figure 1. Global energy investment in clean energy and fossil fuels.

Figure 2. Research design of this paper.

Figure 3. Training–validation–testing data splitting.

Figure 4. Research framework.

Figure 5. Dropout technique in neural networks.

Figure 6. Learning curves of the proposed model.

Figure 7. Comparison of forecasting results and actual observations.

Table 1

Hyperparameter settings of the MTL model.

Model	MTL-1	MTL-2	MTL-3	MTL-4
Epochs	300	300	300	300
Batch size	100	128	150	200
Optimizer	Adam	Adam	Adam	Adam
Initial Learning Rate	0.11	0.11	0.11	0.11
Activation Function	ReLU	ReLU	ReLU	ReLU
Loss Function	${\mathcal{L}}_1$	${\mathcal{L}}_1$	${\mathcal{L}}_1$	${\mathcal{L}}_1$
Dropout	0.05	0.05	0.05	0.05
Weight 1	180	180	180	180
Weight 2	85	85	85	85
Warm-Up	35	35	35	35
Weight Decay	0.003	0.003	0.003	0.003
Early Stop	45	45	45	45

Table 2

Hyperparameter settings of the DT, RF, and XGBoost models.

Model	DT	RF	XGBoost
n_estimators		150	140
max_depth	10	None	7
max_features	None	1
min_samples_leaf	5	3
min_samples_split	10	5
Splitter	best
bootstrap		True
Learning rate			0.09
Colsample_bytree			1.0

Table 3

Hyperparameter settings of the LSTM model.

Model	LSTM
Epochs	300
Batch size	150
Optimizer	Adam
Learning Rate	0.007
Activation Function	ReLU
Layers (LSTM)	5
Neurons (LSTM)	150
Layers (Dense)	2
Neurons (Dense)	50
Dropout	0.2

Table 4

Experimental results in different models.

Models	MAE (kwh)	RMSE (kwh)
MTL-1 (Batch size = 100)	255.77	372.42
MTL-2 (Batch size = 128)	241.49	367.24
MTL-3 (Batch size = 150)	227.99	346.53
MTL-4 (Batch size = 200)	248.52	371.72
Decision Tree	306.32	553.73
Random Forest	263.54	460.94
XGBoost	262.15	465.57
LSTM	3246.41	3816.09

References

1. IEA. World Energy Investment; IEA: Paris, France, 2023; License: CC BY 4.0 Available online: https://www.iea.org/reports/world-energy-investment-2023 (accessed on 21 October 2023).

2. Nti, I.K.; Teimeh, M.; Nyarko-Boateng, O.; Adekoya, A.F. Electricity load forecasting: A systematic review. J. Electr. Syst. Inf. Technol.; 2020; 7, 13. [DOI: https://dx.doi.org/10.1186/s43067-020-00021-8]

3. MOEABOE. Bureau of Energy, Ministry of Economic Affairs. Laws and Regulations. 2021; Available online: https://www.moeaea.gov.tw/ECW/english/home/English.aspx (accessed on 15 November 2023).

4. Guo, S.; Yan, D.; Hu, S.; An, J. Global comparison of building energy use data within the context of climate change. Energy Build.; 2020; 226, 110362. [DOI: https://dx.doi.org/10.1016/j.enbuild.2020.110362]

5. Fiot, J.B.; Dinuzzo, F. Electricity demand forecasting by multi-task learning. IEEE Trans. Smart Grid; 2016; 9, pp. 544-551. [DOI: https://dx.doi.org/10.1109/TSG.2016.2555788]

6. Nalcaci, G.; Özmen, A.; Weber, G.W. Long-term load forecasting: Models based on MARS, ANN and LR methods. Cent. Eur. J. Oper. Res.; 2019; 27, pp. 1033-1049. [DOI: https://dx.doi.org/10.1007/s10100-018-0531-1]

7. Almeshaiei, E.; Soltan, H. A methodology for electric power load forecasting. Alex. Eng. J.; 2011; 50, pp. 137-144. [DOI: https://dx.doi.org/10.1016/j.aej.2011.01.015]

8. Gordillo-Orquera, R.; Lopez-Ramos, L.M.; Muñoz-Romero, S.; Iglesias-Casarrubios, P.; Arcos-Avilés, D.; Marques, A.G.; Rojo-Álvarez, J.L. Analyzing and forecasting electrical load consumption in healthcare buildings. Energies; 2018; 11, 493. [DOI: https://dx.doi.org/10.3390/en11030493]

9. Haque, A.; Rahman, S. Short-term electrical load forecasting through heuristic configuration of regularized deep neural network. Appl. Soft Comput.; 2022; 122, 108877. [DOI: https://dx.doi.org/10.1016/j.asoc.2022.108877]

10. Kyriakides, E.; Polycarpou, M. Short term electric load forecasting: A tutorial. Trends in Neural Computation; Springer: Berlin/Heidelberg, Germany, 2007; pp. 391-418.

11. Chen, P.; Pedersen, T.; Bak-Jensen, B.; Chen, Z. Arima-based time series model of stochastic wind power generation. IEEE Trans. Power Syst.; 2009; 25, pp. 667-676. [DOI: https://dx.doi.org/10.1109/TPWRS.2009.2033277]

12. Musbah, H.; El-Hawary, M. SARIMA Model Forecasting of Short-Term Electrical Load Data Augmented by Fast Fourier Transform Seasonality Detection. Proceedings of the 2019 IEEE Canadian Conference of Electrical and Computer Engineering (CCECE); Edmonton, AB, Canada, 5–8 May 2019.

13. Zhao, Y.; Zhang, C.; Zhang, Y.; Wang, Z.; Li, J. A review of data mining technologies in building energy systems: Load prediction, pattern identification, fault detection and diagnosis. Energy Built Environ.; 2020; 1, pp. 149-164. [DOI: https://dx.doi.org/10.1016/j.enbenv.2019.11.003]

14. Lu, N.; Taylor, T.; Jiang, W.; Jin, C.; Correia, J.; Leung, L.R.; Wong, P.C. Climate change impacts on residential and commercial loads in the western us grid. TEEE Trans. Power Syst.; 2009; 25, pp. 480-488. [DOI: https://dx.doi.org/10.1109/TPWRS.2009.2030387]

15. Rahman, A.; Srikumar, V.; Smith, A.D. Predicting electricity consumption for commercial and residential buildings using deep recurrent neural networks. Appl. Energy; 2018; 212, pp. 372-385. [DOI: https://dx.doi.org/10.1016/j.apenergy.2017.12.051]

16. Toubeau, J.F.; Bottieau, J.; Vallée, F.; De Grève, Z. Deep learning-based multivariate probabilistic forecasting for short-term scheduling in power markets. IEEE Trans. Power Syst.; 2018; 34, pp. 1203-1215. [DOI: https://dx.doi.org/10.1109/TPWRS.2018.2870041]

17. Kong, W.; Dong, Z.Y.; Jia, Y.; Hill, D.J.; Xu, Y.; Zhang, Y. Short-term residential load forecasting based on haviour learning. IEEE Trans. Power Syst.; 2017; 33, pp. 1087-1088. [DOI: https://dx.doi.org/10.1109/TPWRS.2017.2688178]

18. Jalali, S.M.; Ahmadian, S.; Khosravi, A.; Shafie-khah, M.; Nahavandi, S.; Catalão, J.P. A novel evolutionary-based deep convolutional neural network model for intelligent load forecasting. TEEE Trans. Ind. Inform.; 2021; 17, pp. 8243-8253. [DOI: https://dx.doi.org/10.1109/TII.2021.3065718]

19. Dewangan, F.; Abdelaziz, A.Y.; Biswal, M. Load Forecasting Models in Smart Grid Using Smart Meter Information: A Review. Energies; 2023; 16, 1404. [DOI: https://dx.doi.org/10.3390/en16031404]

20. Lam, J.C.; Tsang, C.L.; Yang, L.; Li, D.H. Weather data analysis and design implications for different climatic zones in China. Build. Environ.; 2005; 40, pp. 277-296. [DOI: https://dx.doi.org/10.1016/j.buildenv.2004.07.005]

21. Liu, D.; Wang, W.; Liu, J. Sensitivity analysis of meteorological parameters on building energy consumption. Energy Procedia; 2017; 132, pp. 634-639. [DOI: https://dx.doi.org/10.1016/j.egypro.2017.09.700]

22. Pasion, C.; Wagner, T.; Koschnick, C.; Schuldt, S.; Williams, J.; Hallinan, K. Machine learning modeling of horizontal photovoltaics using weather and location data. Energies; 2020; 13, 2570. [DOI: https://dx.doi.org/10.3390/en13102570]

23. Son, Y.; Yoon, Y.; Cho, J.; Choi, S. Cloud cover forecast based on correlation analysis on satellite images for short-term photovoltaic power forecasting. Sustainability; 2022; 14, 4427. [DOI: https://dx.doi.org/10.3390/su14084427]

24. Fahad, M.U.; Arbab, N. Factor affecting short term load forecasting. J. Clean Energy Technol.; 2014; 2, pp. 305-309. [DOI: https://dx.doi.org/10.7763/JOCET.2014.V2.145]

25. Khatoon, S.; Singh, A.K. Effects of various factors on electric load forecasting: An overview. Proceedings of the 2014 6th IEEE Power India International Conference (PIICON); Delhi, India, 5–7 December 2014; pp. 1-5.

26. Larochelle, H.; Bengio, Y.; Louradour, J.; Lamblin, P. Exploring strategies for training deep neural networks. J. Mach. Learn. Res.; 2009; 10, pp. 1-40.

27. LeCun, Y.; Bengio, Y.; Hinton, G. Deep learning. Nature; 2015; 521, pp. 436-444. [DOI: https://dx.doi.org/10.1038/nature14539] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/26017442]

28. Montavon, G.; Samek, W.; Müller, K.R. Methods for interpreting and understanding deep neural networks. Digit. Signal Process.; 2018; 73, pp. 1-15. [DOI: https://dx.doi.org/10.1016/j.dsp.2017.10.011]

29. Caruana, R. Multitask learning. Mach. Learn.; 1997; 28, pp. 41-75. [DOI: https://dx.doi.org/10.1023/A:1007379606734]

30. Ma, Y.; Yu, J.Q.; Yang, C.Y.; Wang, L. Study on power energy consumption model for large-scale public building. Proceedings of the 2010 2nd International Workshop on Intelligent Systems and Applications; Wuhan, China, 22–23 May 2010; pp. 1-4.

31. Chujai, P.; Kerdprasop, N.; Kerdprasop, K. Time series analysis of household electric consumption with ARIMA and ARMA models. Proceedings of the International Multiconference of Engineers and Computer Scientists; Hong Kong, China, 13–15 March 2013; Volume 1, pp. 295-300.

32. Marín, F.J.; Sandoval, F. Short-term peak load forecasting: Statistical methods versus artificial neural networks. Biological and Artificial Computation: From Neuroscience to Technology, Proceedings of the International Work-Conference on Artificial Neural Networks, Lanzarote, Canary Islands, Spain, 4–6 June 1997; Springer: Berlin/Heidelberg, Germany, 1997; pp. 1334-1343.

33. Zhao, H.X.; Magoulès, F. A review on the prediction of building energy consumption. Renew. Sustain. Energy Rev.; 2012; 16, pp. 3586-3592. [DOI: https://dx.doi.org/10.1016/j.rser.2012.02.049]

34. Ding, N.; Benoit, C.; Foggia, G.; Besanger, Y.; Wurtz, F. Neural network-based model design for short-term load forecast in distribution systems. IEEE Trans. Power Syst.; 2015; 31, pp. 72-81. [DOI: https://dx.doi.org/10.1109/TPWRS.2015.2390132]

35. Hippert, H.S.; Pedreira, C.E.; Souza, R.C. Neural networks for short-term load forecasting: A review and evaluation. IEEE Trans. Power Syst.; 2001; 16, pp. 44-55. [DOI: https://dx.doi.org/10.1109/59.910780]

36. Karatasou, S.; Santamouris, M.; Geros, V. Modeling and predicting building’s energy use with artificial neural networks: Methods and results. Energy Build.; 2006; 38, pp. 949-958. [DOI: https://dx.doi.org/10.1016/j.enbuild.2005.11.005]

37. Xypolytou, E.; Meisel, M.; Sauter, T. Short-term electricity consumption forecast with artificial neural networks—A case study of office buildings. Proceedings of the 2017 IEEE Manchester PowerTech; Manchester, UK, 18–22 June 2017; pp. 1-6.

38. Wen, L.; Zhou, K.; Yang, S. Load demand forecasting of residential buildings using a deep learning model. Electr. Power Syst. Res.; 2020; 179, 106073. [DOI: https://dx.doi.org/10.1016/j.epsr.2019.106073]

39. Dong, S.; Wang, P.; Abbas, K. A survey on deep learning and its applications. Comput. Sci. Rev.; 2021; 40, 100379. [DOI: https://dx.doi.org/10.1016/j.cosrev.2021.100379]

40. Kaiser, K.; Schmid, M.; Schlummer, M. Recycling of polymer-based multilayer packaging: A review. Recycling; 2017; 3, 1. [DOI: https://dx.doi.org/10.3390/recycling3010001]

41. Ruder, S. An overview of multi-task learning in deep neural networks. arXiv; 2017; arXiv: 1706.05098

42. Zhang, L.; Shi, J.; Wang, L.; Xu, C. Electricity, heat, and gas load forecasting based on deep multitask learning in industrial-park integrated energy system. Entropy; 2020; 22, 1355. [DOI: https://dx.doi.org/10.3390/e22121355] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/33266198]

43. Tan, Z.; De, G.; Li, M.; Lin, H.; Yang, S.; Huang, L.; Tan, Q. Combined electricity-heat-cooling-gas load forecasting model for integrated energy system based on multi-task learning and least square support vector machine. J. Clean. Prod.; 2020; 248, 119252. [DOI: https://dx.doi.org/10.1016/j.jclepro.2019.119252]

44. Liu, C.L.; Tseng, C.J.; Huang, T.H.; Yang, J.S.; Huang, K.B. A multi-task learning model for building electrical load prediction. Energy Build.; 2023; 278, 112601. [DOI: https://dx.doi.org/10.1016/j.enbuild.2022.112601]

45. Afrifa-Yamoah, E.; Mueller, U.A.; Taylor, S.M.; Fisher, A.J. Missing data imputation of high-resolution temporal climate time series data. Meteorol. Appl.; 2020; 27, e1873. [DOI: https://dx.doi.org/10.1002/met.1873]

46. Ioffe, S.; Szegedy, C. Batch normalization: Accelerating deep network training by reducing internal covariate shift. Proceedings of the International Conference on Machine Learning; Lille, France, 6–11 July 2015; pp. 448-456.

47. Nair, V.; Hinton, G.E. Rectified linear units improve restricted boltzmann machines. Proceedings of the 27th International Conference on International Conference on Machine Learning, ICML’ 10; Haifa, Israel, 21–24 June 2010; Omnipress: Madison, WI, USA, 2010; pp. 807-814.

48. Nwankpa, C.; Ijomah, W.; Gachagan, A.; Marshall, S. Activation functions: Comparison of trends in practice and research for deep learning. arXiv; 2018; arXiv: 1811.03378

49. Srivastava, N.; Hinton, G.; Krizhevsky, A.; Sutskever, I.; Salakhutdinov, R. Dropout: A simple way to prevent neural networks from overfitting. J. Mach. Learn. Res.; 2014; 15, pp. 1929-1958.

50. Kingma, D.P.; Ba, J. Adam: A method for stochastic optimization. arXiv; 2014; arXiv: 1412.6980

51. Akiba, T.; Sano, S.; Yanase, T.; Ohta, T.; Koyama, M. Optuna: A next-generation hyperparameter optimization framework. Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining; Anchorage, AK, USA, 4–8 August 2019; pp. 2623-2631.

52. Bergstra, J.; Bengio, Y. Random search for hyper-parameter optimization. J. Mach. Learn. Res.; 2012; 13, pp. 281-305.

53. Ozer, I.; Efe, S.B.; Ozbay, H. A combined deep learning application for short term load forecasting. Alex. Eng. J.; 2021; 60, pp. 3807-3818. [DOI: https://dx.doi.org/10.1016/j.aej.2021.02.050]

54. Walker, S.; Khan, W.; Katic, K.; Maassen, W.; Zeiler, W. Accuracy of different machine learning algorithms and added-value of predicting aggregated-level energy performance of commercial buildings. Energy Build.; 2020; 209, 109705. [DOI: https://dx.doi.org/10.1016/j.enbuild.2019.109705]

55. Ghenai, C.; Al-Mufti, O.A.A.; Al-Isawi, O.A.M.; Amirah, L.H.L.; Merabet, A. Short-term building electrical load forecasting using adaptive neuro-fuzzy inference system (ANFIS). J. Build. Eng.; 2022; 52, 104323. [DOI: https://dx.doi.org/10.1016/j.jobe.2022.104323]

56. Ribeiro, A.M.N.; do Carmo, P.R.X.; Endo, P.T.; Rosati, P.; Lynn, T. Short-and very short-term firm-level load forecasting for warehouses: A comparison of machine learning and deep learning models. Energies; 2022; 15, 750. [DOI: https://dx.doi.org/10.3390/en15030750]

57. Wang, X.; Fang, F.; Zhang, X.; Liu, Y.; Wei, L.; Shi, Y. LSTM-based short-term load forecasting for building electricity consumption. Proceedings of the 2019 IEEE 28th International Symposium on Industrial Electronics (ISIE); Vancouver, BC, Canada, 12–14 June 2019; Volume 28, pp. 1418-1423.