Repository: Atik-Ahamed/TimeMachine
Branch: main
Commit: 5bf17a728349
Files: 22
Total size: 81.2 KB
Directory structure:
gitextract_wg48y8vq/
├── .gitignore
├── LICENSE
├── README.md
└── TimeMachine_supervised/
├── RevIN/
│ └── RevIN.py
├── data_provider/
│ ├── data_factory.py
│ └── data_loader.py
├── exp/
│ ├── exp_basic.py
│ └── exp_main.py
├── models/
│ └── TimeMachine.py
├── requirements.txt
├── run_longExp.py
├── scripts/
│ └── TimeMachine/
│ ├── electricity.sh
│ ├── etth1.sh
│ ├── etth2.sh
│ ├── ettm1.sh
│ ├── ettm2.sh
│ ├── traffic.sh
│ └── weather.sh
└── utils/
├── masking.py
├── metrics.py
├── timefeatures.py
└── tools.py
================================================
FILE CONTENTS
================================================
================================================
FILE: .gitignore
================================================
data/
TimeMachine_supervised/exp/__pycache__/
TimeMachine_supervised/data_provider/__pycache__/
TimeMachine_supervised/models/__pycache__/
TimeMachine_supervised/RevIN/__pycache__/
TimeMachine_supervised/utils/__pycache__/
TimeMachine_supervised/checkpoints/
TimeMachine_supervised/logs/LongForecasting/
TimeMachine_supervised/results/
TimeMachine_supervised/test_results/
TimeMachine_supervised/result.txt
TimeMachine_supervised/csv_results/
.vscode/settings.json
================================================
FILE: LICENSE
================================================
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
================================================
FILE: README.md
================================================
# <center>TimeMachine</center>
### Welcome to the official repository of: [TimeMachine: A Time Series is Worth 4 Mambas for Long-term Forecasting](https://arxiv.org/pdf/2403.09898.pdf).
## :triangular_flag_on_post: TimeMachine is accepted to [**ECAI**](https://www.ecai2024.eu/)
## Usage
1. Install requirements. ```pip install -r requirements.txt```
2. Navigate through our example scripts located at ```./scripts/TimeMachine```. You'll find the core of TimeMachine in ```models/TimeMachine.py```. For example, to get the multivariate forecasting results for weather dataset, just run the following command, and you can open ```./result.txt``` to see the results once the training is completed. Moreover, the results will also be available at ```csv_results```, which can be utilized to make queries in the dataframe:
```
sh ./scripts/TimeMachine/weather.sh
```
Hyper-paramters can be tuned based upon needs (e.g. different look-back windows and prediction lengths). TimeMachine is built on the popular [PatchTST](https://github.com/yuqinie98/PatchTST) framework.
## Acknowledgement
We are deeply grateful for the valuable code and efforts contributed by the following GitHub repositories. Their contributions have been immensely beneficial to our work.
- Mamba (https://github.com/state-spaces/mamba)
- PatchTST (https://github.com/yuqinie98/PatchTST)
- iTransformer (https://github.com/thuml/iTransformer)
- RevIN (https://github.com/ts-kim/RevIN)
- Reformer (https://github.com/lucidrains/reformer-pytorch)
- Informer (https://github.com/zhouhaoyi/Informer2020)
- FlashAttention (https://github.com/shreyansh26/FlashAttention-PyTorch)
- Autoformer (https://github.com/thuml/Autoformer)
- Stationary (https://github.com/thuml/Nonstationary_Transformers)
- Time-Series-Library (https://github.com/thuml/Time-Series-Library)
## Citation
If you find this repo useful in your research, please consider citing our paper as follows:
```
@article{timemachine,
title = {TimeMachine: A Time Series is Worth 4 Mambas for Long-term Forecasting},
author = {Ahamed, Md Atik and Cheng, Qiang},
journal = {arXiv preprint arXiv:2403.09898},
year = {2024}
}
```
================================================
FILE: TimeMachine_supervised/RevIN/RevIN.py
================================================
import torch
import torch.nn as nn
class RevIN(nn.Module):
def __init__(self, num_features: int, eps=1e-5, affine=True):
"""
:param num_features: the number of features or channels
:param eps: a value added for numerical stability
:param affine: if True, RevIN has learnable affine parameters
"""
super(RevIN, self).__init__()
self.num_features = num_features
self.eps = eps
self.affine = affine
if self.affine:
self._init_params()
def forward(self, x, mode:str):
if mode == 'norm':
self._get_statistics(x)
x = self._normalize(x)
elif mode == 'denorm':
x = self._denormalize(x)
else: raise NotImplementedError
return x
def _init_params(self):
# initialize RevIN params: (C,)
self.affine_weight = nn.Parameter(torch.ones(self.num_features))
self.affine_bias = nn.Parameter(torch.zeros(self.num_features))
def _get_statistics(self, x):
dim2reduce = tuple(range(1, x.ndim-1))
self.mean = torch.mean(x, dim=dim2reduce, keepdim=True).detach()
self.stdev = torch.sqrt(torch.var(x, dim=dim2reduce, keepdim=True, unbiased=False) + self.eps).detach()
def _normalize(self, x):
x = x - self.mean
x = x / self.stdev
if self.affine:
x = x * self.affine_weight
x = x + self.affine_bias
return x
def _denormalize(self, x):
if self.affine:
x = x - self.affine_bias
x = x / (self.affine_weight + self.eps*self.eps)
x = x * self.stdev
x = x + self.mean
return x
================================================
FILE: TimeMachine_supervised/data_provider/data_factory.py
================================================
from data_provider.data_loader import Dataset_ETT_hour, Dataset_ETT_minute, Dataset_Custom, Dataset_Pred
from torch.utils.data import DataLoader
data_dict = {
'ETTh1': Dataset_ETT_hour,
'ETTh2': Dataset_ETT_hour,
'ETTm1': Dataset_ETT_minute,
'ETTm2': Dataset_ETT_minute,
'custom': Dataset_Custom,
}
def data_provider(args, flag):
Data = data_dict[args.data]
timeenc = 0 if args.embed != 'timeF' else 1
if flag == 'test':
shuffle_flag = False
drop_last = True
batch_size = args.batch_size
freq = args.freq
elif flag == 'pred':
shuffle_flag = False
drop_last = False
batch_size = 1
freq = args.freq
Data = Dataset_Pred
else:
shuffle_flag = True
drop_last = True
batch_size = args.batch_size
freq = args.freq
data_set = Data(
root_path=args.root_path,
data_path=args.data_path,
flag=flag,
size=[args.seq_len, args.label_len, args.pred_len],
features=args.features,
target=args.target,
timeenc=timeenc,
freq=freq
)
print(flag, len(data_set))
data_loader = DataLoader(
data_set,
batch_size=batch_size,
shuffle=shuffle_flag,
num_workers=args.num_workers,
drop_last=drop_last)
return data_set, data_loader
================================================
FILE: TimeMachine_supervised/data_provider/data_loader.py
================================================
import os
import numpy as np
import pandas as pd
import os
import torch
from torch.utils.data import Dataset, DataLoader
from sklearn.preprocessing import StandardScaler
from utils.timefeatures import time_features
import warnings
warnings.filterwarnings('ignore')
class Dataset_ETT_hour(Dataset):
def __init__(self, root_path, flag='train', size=None,
features='S', data_path='ETTh1.csv',
target='OT', scale=True, timeenc=0, freq='h'):
# size [seq_len, label_len, pred_len]
# info
if size == None:
self.seq_len = 24 * 4 * 4
self.label_len = 24 * 4
self.pred_len = 24 * 4
else:
self.seq_len = size[0]
self.label_len = size[1]
self.pred_len = size[2]
# init
assert flag in ['train', 'test', 'val']
type_map = {'train': 0, 'val': 1, 'test': 2}
self.set_type = type_map[flag]
self.features = features
self.target = target
self.scale = scale
self.timeenc = timeenc
self.freq = freq
self.root_path = root_path
self.data_path = data_path
self.__read_data__()
def __read_data__(self):
self.scaler = StandardScaler()
df_raw = pd.read_csv(os.path.join(self.root_path,
self.data_path))
border1s = [0, 12 * 30 * 24 - self.seq_len, 12 * 30 * 24 + 4 * 30 * 24 - self.seq_len]
border2s = [12 * 30 * 24, 12 * 30 * 24 + 4 * 30 * 24, 12 * 30 * 24 + 8 * 30 * 24]
border1 = border1s[self.set_type]
border2 = border2s[self.set_type]
if self.features == 'M' or self.features == 'MS':
cols_data = df_raw.columns[1:]
df_data = df_raw[cols_data]
elif self.features == 'S':
df_data = df_raw[[self.target]]
if self.scale:
train_data = df_data[border1s[0]:border2s[0]]
self.scaler.fit(train_data.values)
data = self.scaler.transform(df_data.values)
else:
data = df_data.values
df_stamp = df_raw[['date']][border1:border2]
df_stamp['date'] = pd.to_datetime(df_stamp.date)
if self.timeenc == 0:
df_stamp['month'] = df_stamp.date.apply(lambda row: row.month, 1)
df_stamp['day'] = df_stamp.date.apply(lambda row: row.day, 1)
df_stamp['weekday'] = df_stamp.date.apply(lambda row: row.weekday(), 1)
df_stamp['hour'] = df_stamp.date.apply(lambda row: row.hour, 1)
data_stamp = df_stamp.drop(['date'], axis=1).values
elif self.timeenc == 1:
data_stamp = time_features(pd.to_datetime(df_stamp['date'].values), freq=self.freq)
data_stamp = data_stamp.transpose(1, 0)
self.data_x = data[border1:border2]
self.data_y = data[border1:border2]
self.data_stamp = data_stamp
def __getitem__(self, index):
s_begin = index
s_end = s_begin + self.seq_len
r_begin = s_end - self.label_len
r_end = r_begin + self.label_len + self.pred_len
seq_x = self.data_x[s_begin:s_end]
seq_y = self.data_y[r_begin:r_end]
seq_x_mark = self.data_stamp[s_begin:s_end]
seq_y_mark = self.data_stamp[r_begin:r_end]
return seq_x, seq_y, seq_x_mark, seq_y_mark
def __len__(self):
return len(self.data_x) - self.seq_len - self.pred_len + 1
def inverse_transform(self, data):
return self.scaler.inverse_transform(data)
class Dataset_ETT_minute(Dataset):
def __init__(self, root_path, flag='train', size=None,
features='S', data_path='ETTm1.csv',
target='OT', scale=True, timeenc=0, freq='t'):
# size [seq_len, label_len, pred_len]
# info
if size == None:
self.seq_len = 24 * 4 * 4
self.label_len = 24 * 4
self.pred_len = 24 * 4
else:
self.seq_len = size[0]
self.label_len = size[1]
self.pred_len = size[2]
# init
assert flag in ['train', 'test', 'val']
type_map = {'train': 0, 'val': 1, 'test': 2}
self.set_type = type_map[flag]
self.features = features
self.target = target
self.scale = scale
self.timeenc = timeenc
self.freq = freq
self.root_path = root_path
self.data_path = data_path
self.__read_data__()
def __read_data__(self):
self.scaler = StandardScaler()
df_raw = pd.read_csv(os.path.join(self.root_path,
self.data_path))
border1s = [0, 12 * 30 * 24 * 4 - self.seq_len, 12 * 30 * 24 * 4 + 4 * 30 * 24 * 4 - self.seq_len]
border2s = [12 * 30 * 24 * 4, 12 * 30 * 24 * 4 + 4 * 30 * 24 * 4, 12 * 30 * 24 * 4 + 8 * 30 * 24 * 4]
border1 = border1s[self.set_type]
border2 = border2s[self.set_type]
if self.features == 'M' or self.features == 'MS':
cols_data = df_raw.columns[1:]
df_data = df_raw[cols_data]
elif self.features == 'S':
df_data = df_raw[[self.target]]
if self.scale:
train_data = df_data[border1s[0]:border2s[0]]
self.scaler.fit(train_data.values)
data = self.scaler.transform(df_data.values)
else:
data = df_data.values
df_stamp = df_raw[['date']][border1:border2]
df_stamp['date'] = pd.to_datetime(df_stamp.date)
if self.timeenc == 0:
df_stamp['month'] = df_stamp.date.apply(lambda row: row.month, 1)
df_stamp['day'] = df_stamp.date.apply(lambda row: row.day, 1)
df_stamp['weekday'] = df_stamp.date.apply(lambda row: row.weekday(), 1)
df_stamp['hour'] = df_stamp.date.apply(lambda row: row.hour, 1)
df_stamp['minute'] = df_stamp.date.apply(lambda row: row.minute, 1)
df_stamp['minute'] = df_stamp.minute.map(lambda x: x // 15)
data_stamp = df_stamp.drop(['date'], axis=1).values
elif self.timeenc == 1:
data_stamp = time_features(pd.to_datetime(df_stamp['date'].values), freq=self.freq)
data_stamp = data_stamp.transpose(1, 0)
self.data_x = data[border1:border2]
self.data_y = data[border1:border2]
self.data_stamp = data_stamp
def __getitem__(self, index):
s_begin = index
s_end = s_begin + self.seq_len
r_begin = s_end - self.label_len
r_end = r_begin + self.label_len + self.pred_len
seq_x = self.data_x[s_begin:s_end]
seq_y = self.data_y[r_begin:r_end]
seq_x_mark = self.data_stamp[s_begin:s_end]
seq_y_mark = self.data_stamp[r_begin:r_end]
return seq_x, seq_y, seq_x_mark, seq_y_mark
def __len__(self):
return len(self.data_x) - self.seq_len - self.pred_len + 1
def inverse_transform(self, data):
return self.scaler.inverse_transform(data)
class Dataset_Custom(Dataset):
def __init__(self, root_path, flag='train', size=None,
features='S', data_path='ETTh1.csv',
target='OT', scale=True, timeenc=0, freq='h'):
# size [seq_len, label_len, pred_len]
# info
if size == None:
self.seq_len = 24 * 4 * 4
self.label_len = 24 * 4
self.pred_len = 24 * 4
else:
self.seq_len = size[0]
self.label_len = size[1]
self.pred_len = size[2]
# init
assert flag in ['train', 'test', 'val']
type_map = {'train': 0, 'val': 1, 'test': 2}
self.set_type = type_map[flag]
self.features = features
self.target = target
self.scale = scale
self.timeenc = timeenc
self.freq = freq
self.root_path = root_path
self.data_path = data_path
self.__read_data__()
def __read_data__(self):
self.scaler = StandardScaler()
df_raw = pd.read_csv(os.path.join(self.root_path,
self.data_path))
'''
df_raw.columns: ['date', ...(other features), target feature]
'''
cols = list(df_raw.columns)
cols.remove(self.target)
cols.remove('date')
df_raw = df_raw[['date'] + cols + [self.target]]
# print(cols)
num_train = int(len(df_raw) * 0.7)
num_test = int(len(df_raw) * 0.2)
num_vali = len(df_raw) - num_train - num_test
border1s = [0, num_train - self.seq_len, len(df_raw) - num_test - self.seq_len]
border2s = [num_train, num_train + num_vali, len(df_raw)]
border1 = border1s[self.set_type]
border2 = border2s[self.set_type]
if self.features == 'M' or self.features == 'MS':
cols_data = df_raw.columns[1:]
df_data = df_raw[cols_data]
elif self.features == 'S':
df_data = df_raw[[self.target]]
if self.scale:
train_data = df_data[border1s[0]:border2s[0]]
self.scaler.fit(train_data.values)
# print(self.scaler.mean_)
# exit()
data = self.scaler.transform(df_data.values)
else:
data = df_data.values
df_stamp = df_raw[['date']][border1:border2]
df_stamp['date'] = pd.to_datetime(df_stamp.date)
if self.timeenc == 0:
df_stamp['month'] = df_stamp.date.apply(lambda row: row.month, 1)
df_stamp['day'] = df_stamp.date.apply(lambda row: row.day, 1)
df_stamp['weekday'] = df_stamp.date.apply(lambda row: row.weekday(), 1)
df_stamp['hour'] = df_stamp.date.apply(lambda row: row.hour, 1)
data_stamp = df_stamp.drop(['date'], axis=1).values
elif self.timeenc == 1:
data_stamp = time_features(pd.to_datetime(df_stamp['date'].values), freq=self.freq)
data_stamp = data_stamp.transpose(1, 0)
self.data_x = data[border1:border2]
self.data_y = data[border1:border2]
self.data_stamp = data_stamp
def __getitem__(self, index):
s_begin = index
s_end = s_begin + self.seq_len
r_begin = s_end - self.label_len
r_end = r_begin + self.label_len + self.pred_len
seq_x = self.data_x[s_begin:s_end]
seq_y = self.data_y[r_begin:r_end]
seq_x_mark = self.data_stamp[s_begin:s_end]
seq_y_mark = self.data_stamp[r_begin:r_end]
return seq_x, seq_y, seq_x_mark, seq_y_mark
def __len__(self):
return len(self.data_x) - self.seq_len - self.pred_len + 1
def inverse_transform(self, data):
return self.scaler.inverse_transform(data)
class Dataset_Pred(Dataset):
def __init__(self, root_path, flag='pred', size=None,
features='S', data_path='ETTh1.csv',
target='OT', scale=True, inverse=False, timeenc=0, freq='15min', cols=None):
# size [seq_len, label_len, pred_len]
# info
if size == None:
self.seq_len = 24 * 4 * 4
self.label_len = 24 * 4
self.pred_len = 24 * 4
else:
self.seq_len = size[0]
self.label_len = size[1]
self.pred_len = size[2]
# init
assert flag in ['pred']
self.features = features
self.target = target
self.scale = scale
self.inverse = inverse
self.timeenc = timeenc
self.freq = freq
self.cols = cols
self.root_path = root_path
self.data_path = data_path
self.__read_data__()
def __read_data__(self):
self.scaler = StandardScaler()
df_raw = pd.read_csv(os.path.join(self.root_path,
self.data_path))
'''
df_raw.columns: ['date', ...(other features), target feature]
'''
if self.cols:
cols = self.cols.copy()
cols.remove(self.target)
else:
cols = list(df_raw.columns)
cols.remove(self.target)
cols.remove('date')
df_raw = df_raw[['date'] + cols + [self.target]]
border1 = len(df_raw) - self.seq_len
border2 = len(df_raw)
if self.features == 'M' or self.features == 'MS':
cols_data = df_raw.columns[1:]
df_data = df_raw[cols_data]
elif self.features == 'S':
df_data = df_raw[[self.target]]
if self.scale:
self.scaler.fit(df_data.values)
data = self.scaler.transform(df_data.values)
else:
data = df_data.values
tmp_stamp = df_raw[['date']][border1:border2]
tmp_stamp['date'] = pd.to_datetime(tmp_stamp.date)
pred_dates = pd.date_range(tmp_stamp.date.values[-1], periods=self.pred_len + 1, freq=self.freq)
df_stamp = pd.DataFrame(columns=['date'])
df_stamp.date = list(tmp_stamp.date.values) + list(pred_dates[1:])
if self.timeenc == 0:
df_stamp['month'] = df_stamp.date.apply(lambda row: row.month, 1)
df_stamp['day'] = df_stamp.date.apply(lambda row: row.day, 1)
df_stamp['weekday'] = df_stamp.date.apply(lambda row: row.weekday(), 1)
df_stamp['hour'] = df_stamp.date.apply(lambda row: row.hour, 1)
df_stamp['minute'] = df_stamp.date.apply(lambda row: row.minute, 1)
df_stamp['minute'] = df_stamp.minute.map(lambda x: x // 15)
data_stamp = df_stamp.drop(['date'], axis=1).values
elif self.timeenc == 1:
data_stamp = time_features(pd.to_datetime(df_stamp['date'].values), freq=self.freq)
data_stamp = data_stamp.transpose(1, 0)
self.data_x = data[border1:border2]
if self.inverse:
self.data_y = df_data.values[border1:border2]
else:
self.data_y = data[border1:border2]
self.data_stamp = data_stamp
def __getitem__(self, index):
s_begin = index
s_end = s_begin + self.seq_len
r_begin = s_end - self.label_len
r_end = r_begin + self.label_len + self.pred_len
seq_x = self.data_x[s_begin:s_end]
if self.inverse:
seq_y = self.data_x[r_begin:r_begin + self.label_len]
else:
seq_y = self.data_y[r_begin:r_begin + self.label_len]
seq_x_mark = self.data_stamp[s_begin:s_end]
seq_y_mark = self.data_stamp[r_begin:r_end]
return seq_x, seq_y, seq_x_mark, seq_y_mark
def __len__(self):
return len(self.data_x) - self.seq_len + 1
def inverse_transform(self, data):
return self.scaler.inverse_transform(data)
================================================
FILE: TimeMachine_supervised/exp/exp_basic.py
================================================
import os
import torch
import numpy as np
class Exp_Basic(object):
def __init__(self, args):
self.args = args
self.device = self._acquire_device()
self.model = self._build_model().to(self.device)
def _build_model(self):
raise NotImplementedError
return None
def _acquire_device(self):
if self.args.use_gpu:
os.environ["CUDA_VISIBLE_DEVICES"] = str(
self.args.gpu) if not self.args.use_multi_gpu else self.args.devices
device = torch.device('cuda:{}'.format(self.args.gpu))
print('Use GPU: cuda:{}'.format(self.args.gpu))
else:
device = torch.device('cpu')
print('Use CPU')
return device
def _get_data(self):
pass
def vali(self):
pass
def train(self):
pass
def test(self):
pass
================================================
FILE: TimeMachine_supervised/exp/exp_main.py
================================================
from data_provider.data_factory import data_provider
from exp.exp_basic import Exp_Basic
from models import TimeMachine
from utils.tools import EarlyStopping, adjust_learning_rate, visual, test_params_flop
from utils.metrics import metric
import numpy as np
import torch
import torch.nn as nn
from torch import optim
from torch.optim import lr_scheduler
import pandas as pd
import os
import time
import warnings
import matplotlib.pyplot as plt
import numpy as np
warnings.filterwarnings('ignore')
class Exp_Main(Exp_Basic):
def __init__(self, args):
super(Exp_Main, self).__init__(args)
def _build_model(self):
model_dict = {
'TimeMachine':TimeMachine
}
model = model_dict[self.args.model].Model(self.args).float()
if self.args.use_multi_gpu and self.args.use_gpu:
model = nn.DataParallel(model, device_ids=self.args.device_ids)
return model
def _get_data(self, flag):
data_set, data_loader = data_provider(self.args, flag)
return data_set, data_loader
def _select_optimizer(self):
model_optim = optim.Adam(self.model.parameters(), lr=self.args.learning_rate)
return model_optim
def _select_criterion(self):
criterion = nn.MSELoss()
return criterion
def vali(self, vali_data, vali_loader, criterion):
total_loss = []
self.model.eval()
with torch.no_grad():
for i, (batch_x, batch_y, batch_x_mark, batch_y_mark) in enumerate(vali_loader):
batch_x = batch_x.float().to(self.device)
batch_y = batch_y.float()
batch_x_mark = batch_x_mark.float().to(self.device)
batch_y_mark = batch_y_mark.float().to(self.device)
# decoder input
dec_inp = torch.zeros_like(batch_y[:, -self.args.pred_len:, :]).float()
dec_inp = torch.cat([batch_y[:, :self.args.label_len, :], dec_inp], dim=1).float().to(self.device)
# encoder - decoder
if self.args.use_amp:
with torch.cuda.amp.autocast():
if 'Machine' in self.args.model:
outputs = self.model(batch_x)
else:
if 'Machine' in self.args.model:
outputs = self.model(batch_x)
f_dim = -1 if self.args.features == 'MS' else 0
outputs = outputs[:, -self.args.pred_len:, f_dim:]
batch_y = batch_y[:, -self.args.pred_len:, f_dim:].to(self.device)
pred = outputs.detach().cpu()
true = batch_y.detach().cpu()
loss = criterion(pred, true)
total_loss.append(loss)
total_loss = np.average(total_loss)
self.model.train()
return total_loss
def train(self, setting):
train_data, train_loader = self._get_data(flag='train')
vali_data, vali_loader = self._get_data(flag='val')
test_data, test_loader = self._get_data(flag='test')
path = os.path.join(self.args.checkpoints, setting)
if not os.path.exists(path):
os.makedirs(path)
time_now = time.time()
train_steps = len(train_loader)
early_stopping = EarlyStopping(patience=self.args.patience, verbose=True)
model_optim = self._select_optimizer()
criterion = self._select_criterion()
if self.args.use_amp:
scaler = torch.cuda.amp.GradScaler()
scheduler = lr_scheduler.OneCycleLR(optimizer = model_optim,
steps_per_epoch = train_steps,
pct_start = self.args.pct_start,
epochs = self.args.train_epochs,
max_lr = self.args.learning_rate)
for epoch in range(self.args.train_epochs):
iter_count = 0
train_loss = []
self.model.train()
epoch_time = time.time()
for i, (batch_x, batch_y, batch_x_mark, batch_y_mark) in enumerate(train_loader):
iter_count += 1
model_optim.zero_grad()
batch_x = batch_x.float().to(self.device)
batch_y = batch_y.float().to(self.device)
batch_x_mark = batch_x_mark.float().to(self.device)
batch_y_mark = batch_y_mark.float().to(self.device)
# decoder input
dec_inp = torch.zeros_like(batch_y[:, -self.args.pred_len:, :]).float()
dec_inp = torch.cat([batch_y[:, :self.args.label_len, :], dec_inp], dim=1).float().to(self.device)
# encoder - decoder
if self.args.use_amp:
with torch.cuda.amp.autocast():
if 'Machine' in self.args.model:
outputs = self.model(batch_x)
f_dim = -1 if self.args.features == 'MS' else 0
outputs = outputs[:, -self.args.pred_len:, f_dim:]
batch_y = batch_y[:, -self.args.pred_len:, f_dim:].to(self.device)
loss = criterion(outputs, batch_y)
train_loss.append(loss.item())
else:
if 'Machine' in self.args.model:
outputs = self.model(batch_x)
# print(outputs.shape,batch_y.shape)
f_dim = -1 if self.args.features == 'MS' else 0
outputs = outputs[:, -self.args.pred_len:, f_dim:]
batch_y = batch_y[:, -self.args.pred_len:, f_dim:].to(self.device)
loss = criterion(outputs, batch_y)
train_loss.append(loss.item())
if (i + 1) % 100 == 0:
print("\titers: {0}, epoch: {1} | loss: {2:.7f}".format(i + 1, epoch + 1, loss.item()))
speed = (time.time() - time_now) / iter_count
left_time = speed * ((self.args.train_epochs - epoch) * train_steps - i)
print('\tspeed: {:.4f}s/iter; left time: {:.4f}s'.format(speed, left_time))
iter_count = 0
time_now = time.time()
if self.args.use_amp:
scaler.scale(loss).backward()
scaler.step(model_optim)
scaler.update()
else:
loss.backward()
model_optim.step()
if self.args.lradj == 'TST':
adjust_learning_rate(model_optim, scheduler, epoch + 1, self.args, printout=False)
scheduler.step()
print("Epoch: {} cost time: {}".format(epoch + 1, time.time() - epoch_time))
train_loss = np.average(train_loss)
vali_loss = self.vali(vali_data, vali_loader, criterion)
test_loss = self.vali(test_data, test_loader, criterion)
print("Epoch: {0}, Steps: {1} | Train Loss: {2:.7f} Vali Loss: {3:.7f} Test Loss: {4:.7f}".format(
epoch + 1, train_steps, train_loss, vali_loss, test_loss))
early_stopping(vali_loss, self.model, path)
if early_stopping.early_stop:
print("Early stopping")
break
if self.args.lradj != 'TST':
adjust_learning_rate(model_optim, scheduler, epoch + 1, self.args)
else:
print('Updating learning rate to {}'.format(scheduler.get_last_lr()[0]))
best_model_path = path + '/' + 'checkpoint.pth'
self.model.load_state_dict(torch.load(best_model_path))
return self.model
def test(self, setting, test=0):
test_data, test_loader = self._get_data(flag='test')
if test:
print('loading model')
self.model.load_state_dict(torch.load(os.path.join('./checkpoints/' + setting, 'checkpoint.pth')))
preds = []
trues = []
inputx = []
folder_path = './test_results/' + setting + '/'
if not os.path.exists(folder_path):
os.makedirs(folder_path)
self.model.eval()
with torch.no_grad():
for i, (batch_x, batch_y, batch_x_mark, batch_y_mark) in enumerate(test_loader):
batch_x = batch_x.float().to(self.device)
batch_y = batch_y.float().to(self.device)
batch_x_mark = batch_x_mark.float().to(self.device)
batch_y_mark = batch_y_mark.float().to(self.device)
# decoder input
dec_inp = torch.zeros_like(batch_y[:, -self.args.pred_len:, :]).float()
dec_inp = torch.cat([batch_y[:, :self.args.label_len, :], dec_inp], dim=1).float().to(self.device)
# encoder - decoder
if self.args.use_amp:
with torch.cuda.amp.autocast():
if 'Machine' in self.args.model:
outputs = self.model(batch_x)
else:
if 'Machine' in self.args.model:
outputs = self.model(batch_x)
f_dim = -1 if self.args.features == 'MS' else 0
# print(outputs.shape,batch_y.shape)
outputs = outputs[:, -self.args.pred_len:, f_dim:]
batch_y = batch_y[:, -self.args.pred_len:, f_dim:].to(self.device)
outputs = outputs.detach().cpu().numpy()
batch_y = batch_y.detach().cpu().numpy()
pred = outputs # outputs.detach().cpu().numpy() # .squeeze()
true = batch_y # batch_y.detach().cpu().numpy() # .squeeze()
preds.append(pred)
trues.append(true)
inputx.append(batch_x.detach().cpu().numpy())
if i % 20 == 0:
input = batch_x.detach().cpu().numpy()
gt = np.concatenate((input[0, :, -1], true[0, :, -1]), axis=0)
prd = np.concatenate((input[0, :, -1], pred[0, :, -1]), axis=0)
visual(gt, prd, os.path.join(folder_path, str(i) + '.pdf'))
if self.args.test_flop:
test_params_flop((batch_x.shape[1],batch_x.shape[2]))
exit()
preds = np.array(preds)
trues = np.array(trues)
inputx = np.array(inputx)
preds = preds.reshape(-1, preds.shape[-2], preds.shape[-1])
trues = trues.reshape(-1, trues.shape[-2], trues.shape[-1])
inputx = inputx.reshape(-1, inputx.shape[-2], inputx.shape[-1])
# result save
folder_path = './results/' + setting + '/'
if not os.path.exists(folder_path):
os.makedirs(folder_path)
mae, mse, rmse, mape, mspe, rse, corr = metric(preds, trues)
print('mse:{}, mae:{}, rse:{}'.format(mse, mae, rse))
f = open("result.txt", 'a')
f.write(setting + " \n")
f.write('mse:{}, mae:{}, rse:{}'.format(mse, mae, rse))
f.write('\n')
f.write('\n')
f.close()
temp_df = pd.DataFrame()
temp_df['Seed']=[self.args.random_seed]
temp_df['Model']=[self.args.model]
temp_df['seq_len']=[self.args.seq_len]
temp_df['label_len']=[self.args.label_len]
temp_df['pred_len']=[self.args.pred_len]
temp_df['n1']=[self.args.n1]
temp_df['n2']=[self.args.n2]
temp_df['dropout']=[self.args.dropout]
temp_df['train_epochs']=[self.args.train_epochs]
temp_df['batch']=[self.args.batch_size]
temp_df['patience']=[self.args.patience]
temp_df['LR']=[self.args.learning_rate]
temp_df['dropout']=[self.args.dropout]
temp_df['ch_ind']=[self.args.ch_ind]
temp_df['revin']=[self.args.revin]
temp_df['e_fact']=[self.args.e_fact]
temp_df['dconv']=[self.args.dconv]
temp_df['MSE']=[mse]
temp_df['MAE']=[mae]
temp_df['residual']=[self.args.residual]
temp_df['d_state']=[self.args.d_state]
temp_df['checkpoint_path']=[setting]
if not os.path.exists('./csv_results/'+'result_'+self.args.data_path):
temp_df.to_csv('./csv_results/'+'result_'+self.args.data_path, index=False)
else:
result_df=pd.read_csv('./csv_results/'+'result_'+self.args.data_path)
result_df = pd.concat([result_df,temp_df],ignore_index=True)
result_df.to_csv('./csv_results/'+'result_'+self.args.data_path, index=False)
# np.save(folder_path + 'metrics.npy', np.array([mae, mse, rmse, mape, mspe,rse, corr]))
np.save(folder_path + 'pred.npy', preds)
np.save(folder_path + 'true.npy', trues)
np.save(folder_path + 'x.npy', inputx)
return
def predict(self, setting, load=False):
pred_data, pred_loader = self._get_data(flag='pred')
if load:
path = os.path.join(self.args.checkpoints, setting)
best_model_path = path + '/' + 'checkpoint.pth'
self.model.load_state_dict(torch.load(best_model_path))
preds = []
self.model.eval()
with torch.no_grad():
for i, (batch_x, batch_y, batch_x_mark, batch_y_mark) in enumerate(pred_loader):
batch_x = batch_x.float().to(self.device)
batch_y = batch_y.float()
batch_x_mark = batch_x_mark.float().to(self.device)
batch_y_mark = batch_y_mark.float().to(self.device)
# decoder input
dec_inp = torch.zeros([batch_y.shape[0], self.args.pred_len, batch_y.shape[2]]).float().to(batch_y.device)
dec_inp = torch.cat([batch_y[:, :self.args.label_len, :], dec_inp], dim=1).float().to(self.device)
# encoder - decoder
if self.args.use_amp:
with torch.cuda.amp.autocast():
if 'Machine' in self.args.model:
outputs = self.model(batch_x)
else:
if 'Machine' in self.args.model:
outputs = self.model(batch_x)
pred = outputs.detach().cpu().numpy() # .squeeze()
preds.append(pred)
preds = np.array(preds)
preds = preds.reshape(-1, preds.shape[-2], preds.shape[-1])
# result save
folder_path = './results/' + setting + '/'
if not os.path.exists(folder_path):
os.makedirs(folder_path)
np.save(folder_path + 'real_prediction.npy', preds)
return
================================================
FILE: TimeMachine_supervised/models/TimeMachine.py
================================================
import torch
from mamba_ssm import Mamba
from RevIN.RevIN import RevIN
class Model(torch.nn.Module):
def __init__(self,configs):
super(Model, self).__init__()
self.configs=configs
if self.configs.revin==1:
self.revin_layer = RevIN(self.configs.enc_in)
self.lin1=torch.nn.Linear(self.configs.seq_len,self.configs.n1)
self.dropout1=torch.nn.Dropout(self.configs.dropout)
self.lin2=torch.nn.Linear(self.configs.n1,self.configs.n2)
self.dropout2=torch.nn.Dropout(self.configs.dropout)
if self.configs.ch_ind==1:
self.d_model_param1=1
self.d_model_param2=1
else:
self.d_model_param1=self.configs.n2
self.d_model_param2=self.configs.n1
self.mamba1=Mamba(d_model=self.d_model_param1,d_state=self.configs.d_state,d_conv=self.configs.dconv,expand=self.configs.e_fact)
self.mamba2=Mamba(d_model=self.configs.n2,d_state=self.configs.d_state,d_conv=self.configs.dconv,expand=self.configs.e_fact)
self.mamba3=Mamba(d_model=self.configs.n1,d_state=self.configs.d_state,d_conv=self.configs.dconv,expand=self.configs.e_fact)
self.mamba4=Mamba(d_model=self.d_model_param2,d_state=self.configs.d_state,d_conv=self.configs.dconv,expand=self.configs.e_fact)
self.lin3=torch.nn.Linear(self.configs.n2,self.configs.n1)
self.lin4=torch.nn.Linear(2*self.configs.n1,self.configs.pred_len)
def forward(self, x):
if self.configs.revin==1:
x=self.revin_layer(x,'norm')
else:
means = x.mean(1, keepdim=True).detach()
x = x - means
stdev = torch.sqrt(torch.var(x, dim=1, keepdim=True, unbiased=False) + 1e-5)
x /= stdev
x=torch.permute(x,(0,2,1))
if self.configs.ch_ind==1:
x=torch.reshape(x,(x.shape[0]*x.shape[1],1,x.shape[2]))
x=self.lin1(x)
x_res1=x
x=self.dropout1(x)
x3=self.mamba3(x)
if self.configs.ch_ind==1:
x4=torch.permute(x,(0,2,1))
else:
x4=x
x4=self.mamba4(x4)
if self.configs.ch_ind==1:
x4=torch.permute(x4,(0,2,1))
x4=x4+x3
x=self.lin2(x)
x_res2=x
x=self.dropout2(x)
if self.configs.ch_ind==1:
x1=torch.permute(x,(0,2,1))
else:
x1=x
x1=self.mamba1(x1)
if self.configs.ch_ind==1:
x1=torch.permute(x1,(0,2,1))
x2=self.mamba2(x)
if self.configs.residual==1:
x=x1+x_res2+x2
else:
x=x1+x2
x=self.lin3(x)
if self.configs.residual==1:
x=x+x_res1
x=torch.cat([x,x4],dim=2)
x=self.lin4(x)
if self.configs.ch_ind==1:
x=torch.reshape(x,(-1,self.configs.enc_in,self.configs.pred_len))
x=torch.permute(x,(0,2,1))
if self.configs.revin==1:
x=self.revin_layer(x,'denorm')
else:
x = x * (stdev[:, 0, :].unsqueeze(1).repeat(1, self.configs.pred_len, 1))
x = x + (means[:, 0, :].unsqueeze(1).repeat(1, self.configs.pred_len, 1))
return x
================================================
FILE: TimeMachine_supervised/requirements.txt
================================================
numpy
matplotlib
pandas
scikit-learn
torch
mamba-ssm
causal-conv1d>=1.2.0
================================================
FILE: TimeMachine_supervised/run_longExp.py
================================================
import argparse
import os
import torch
from exp.exp_main import Exp_Main
import random
import numpy as np
if __name__ == '__main__':
parser = argparse.ArgumentParser(description='Time Series Forecasting')
# RANDOM SEED
parser.add_argument('--random_seed', type=int, default=2021, help='random seed')
# BASIC CONFIG
parser.add_argument('--is_training', type=int, required=True, default=1, help='status')
parser.add_argument('--model_id', type=str, required=True, default='test', help='model id')
parser.add_argument('--model', type=str, required=True, default='Autoformer',
help='model name, options: [TimeMachine]')
parser.add_argument('--model_id_name', type=str, required=False, default='custom', help='model id name')
# DATALOADER
parser.add_argument('--data', type=str, required=True, default='ETTm1', help='dataset type')
parser.add_argument('--root_path', type=str, default='./data/ETT/', help='root path of the data file')
parser.add_argument('--data_path', type=str, default='ETTh1.csv', help='data file')
parser.add_argument('--features', type=str, default='M',
help='forecasting task, options:[M, S, MS]; M:multivariate predict multivariate, S:univariate predict univariate, MS:multivariate predict univariate')
parser.add_argument('--target', type=str, default='OT', help='target feature in S or MS task')
parser.add_argument('--freq', type=str, default='h',
help='freq for time features encoding, options:[s:secondly, t:minutely, h:hourly, d:daily, b:business days, w:weekly, m:monthly], you can also use more detailed freq like 15min or 3h')
parser.add_argument('--checkpoints', type=str, default='./checkpoints/', help='location of model checkpoints')
# FORECASTING TASK
parser.add_argument('--seq_len', type=int, default=96, help='input sequence length')
parser.add_argument('--label_len', type=int, default=48, help='start token length')
parser.add_argument('--pred_len', type=int, default=96, help='prediction sequence length')
parser.add_argument('--n1',type=int,default=256,help='First Embedded representation')
parser.add_argument('--n2',type=int,default=128,help='Second Embedded representation')
# METHOD
parser.add_argument('--revin', type=int, default=1, help='RevIN; True 1 False 0')
parser.add_argument('--ch_ind', type=int, default=1, help='Channel Independence; True 1 False 0')
parser.add_argument('--residual', type=int, default=1, help='Residual Connection; True 1 False 0')
parser.add_argument('--d_state', type=int, default=256, help='d_state parameter of Mamba')
parser.add_argument('--dconv', type=int, default=2, help='d_conv parameter of Mamba')
parser.add_argument('--e_fact', type=int, default=1, help='expand factor parameter of Mamba')
parser.add_argument('--enc_in', type=int, default=7, help='encoder input size') #Use this hyperparameter as the number of channels
parser.add_argument('--dropout', type=float, default=0.05, help='dropout')
parser.add_argument('--embed', type=str, default='timeF',
help='time features encoding, options:[timeF, fixed, learned]')
parser.add_argument('--do_predict', action='store_true', help='whether to predict unseen future data')
# OPTIMIZATION
parser.add_argument('--num_workers', type=int, default=10, help='data loader num workers')
parser.add_argument('--itr', type=int, default=2, help='experiments times')
parser.add_argument('--train_epochs', type=int, default=100, help='train epochs')
parser.add_argument('--batch_size', type=int, default=16, help='batch size of train input data')
parser.add_argument('--patience', type=int, default=100, help='early stopping patience')
parser.add_argument('--learning_rate', type=float, default=0.0001, help='optimizer learning rate')
parser.add_argument('--des', type=str, default='test', help='exp description')
parser.add_argument('--loss', type=str, default='mse', help='loss function')
parser.add_argument('--lradj', type=str, default='type3', help='adjust learning rate')
parser.add_argument('--pct_start', type=float, default=0.3, help='pct_start')
parser.add_argument('--use_amp', action='store_true', help='use automatic mixed precision training', default=False)
# GPU
parser.add_argument('--use_gpu', type=bool, default=True, help='use gpu')
parser.add_argument('--gpu', type=int, default=0, help='gpu')
parser.add_argument('--use_multi_gpu', action='store_true', help='use multiple gpus', default=False)
parser.add_argument('--devices', type=str, default='0,1,2,3', help='device ids of multile gpus')
parser.add_argument('--test_flop', action='store_true', default=False, help='See utils/tools for usage')
args = parser.parse_args()
# random seed
fix_seed = args.random_seed
random.seed(fix_seed)
torch.manual_seed(fix_seed)
np.random.seed(fix_seed)
args.model_id_name=args.data_path[:-4]
args.use_gpu = True if torch.cuda.is_available() and args.use_gpu else False
if args.use_gpu and args.use_multi_gpu:
args.dvices = args.devices.replace(' ', '')
device_ids = args.devices.split(',')
args.device_ids = [int(id_) for id_ in device_ids]
args.gpu = args.device_ids[0]
print('Args in experiment:')
print(args)
Exp = Exp_Main
if args.is_training:
for ii in range(args.itr):
# setting record of experiments
setting = '{}_{}_{}_ft{}_sl{}_ll{}_pl{}_n1{}_n2{}_dr{}_cin{}_rin{}_res{}_dst{}_dconv{}_efact{}'.format(
args.model_id,
args.model,
args.model_id_name,
args.features,
args.seq_len,
args.label_len,
args.pred_len,
args.n1,
args.n2,
args.dropout,
args.ch_ind,
args.revin,
args.residual,
args.d_state,
args.dconv,
args.e_fact)
exp = Exp(args) # set experiments
print('>>>>>>>start training : {}>>>>>>>>>>>>>>>>>>>>>>>>>>'.format(setting))
exp.train(setting)
print('>>>>>>>testing : {}<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<'.format(setting))
exp.test(setting)
if args.do_predict:
print('>>>>>>>predicting : {}<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<'.format(setting))
exp.predict(setting, True)
torch.cuda.empty_cache()
else:
ii = 0
setting = '{}_{}_{}_ft{}_sl{}_ll{}_pl{}_n1{}_n2{}_dr{}_cin{}_rin{}_res{}_dst{}_dconv{}_efact'.format(
args.model_id,
args.model,
args.model_id_name,
args.features,
args.seq_len,
args.label_len,
args.pred_len,
args.n1,
args.n2,
args.dropout,
args.ch_ind,
args.revin,
args.residual,
args.d_state,
args.dconv,
args.e_fact)
exp = Exp(args) # set experiments
print('>>>>>>>testing : {}<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<'.format(setting))
exp.test(setting, test=1)
torch.cuda.empty_cache()
================================================
FILE: TimeMachine_supervised/scripts/TimeMachine/electricity.sh
================================================
if [ ! -d "./logs" ]; then
mkdir ./logs
fi
if [ ! -d "./logs/LongForecasting" ]; then
mkdir ./logs/LongForecasting
fi
if [ ! -d "./csv_results" ]; then
mkdir ./csv_results
fi
if [ ! -d "./results" ]; then
mkdir ./results
fi
if [ ! -d "./test_results" ]; then
mkdir ./test_results
fi
model_name=TimeMachine
root_path_name=../data/electricity
data_path_name=electricity.csv
model_id_name=electricity
data_name=custom
rin=1
random_seed=2024
one=96
two=192
three=336
four=720
residual=1
fc_drop=0.0
dstate=256
dconv=2
for seq_len in 96
do
for pred_len in 96 192 336 720
do
for e_fact in 1
do
if [ $pred_len -eq $one ]
then
n1=512
n2=128
fc_drop=0.0
fi
if [ $pred_len -eq $two ]
then
n1=512
n2=256
fc_drop=0.0
fi
if [ $pred_len -eq $three ]
then
n1=512
n2=256
fc_drop=0.4
fi
if [ $pred_len -eq $four ]
then
n1=512
n2=8
fc_drop=0.0
fi
python -u run_longExp.py \
--random_seed $random_seed \
--is_training 1 \
--root_path $root_path_name \
--data_path $data_path_name \
--model_id $model_id_name_$seq_len'_'$pred_len \
--model $model_name \
--data $data_name \
--features M \
--seq_len $seq_len \
--pred_len $pred_len \
--enc_in 321 \
--n1 $n1 \
--n2 $n2 \
--dropout $fc_drop\
--revin 1\
--ch_ind 0\
--residual $residual\
--dconv $dconv \
--d_state $dstate\
--e_fact $e_fact\
--des 'Exp' \
--lradj '5' \
--pct_start 0.2 \
--train_epochs 100\
--itr 1 --batch_size 16 --learning_rate 0.001 >logs/LongForecasting/$model_name'_'$model_id_name'_'$seq_len'_'$pred_len'_'$n1'_'$n2'_'$fc_drop'_'$rin'_'$residual'_'$dstate'_'$dconv'_'$e_fact.log
done
done
done
================================================
FILE: TimeMachine_supervised/scripts/TimeMachine/etth1.sh
================================================
if [ ! -d "./logs" ]; then
mkdir ./logs
fi
if [ ! -d "./logs/LongForecasting" ]; then
mkdir ./logs/LongForecasting
fi
if [ ! -d "./csv_results" ]; then
mkdir ./csv_results
fi
if [ ! -d "./results" ]; then
mkdir ./results
fi
if [ ! -d "./test_results" ]; then
mkdir ./test_results
fi
model_name=TimeMachine
root_path_name=../data/ETT-small
data_path_name=ETTh1.csv
model_id_name=ETTh1
data_name=ETTh1
rin=1
random_seed=2024
one=96
two=192
three=336
four=720
residual=1
fc_drop=0.7
dstate=256
dconv=2
for seq_len in 96
do
for pred_len in 96 192 336 720
do
for e_fact in 1
do
if [ $pred_len -eq $one ]
then
n1=512
n2=32
fi
if [ $pred_len -eq $two ]
then
n1=512
n2=64
fi
if [ $pred_len -eq $three ]
then
n1=512
n2=128
fi
if [ $pred_len -eq $four ]
then
n1=128
n2=16
fi
python -u run_longExp.py \
--random_seed $random_seed \
--is_training 1 \
--root_path $root_path_name \
--data_path $data_path_name \
--model_id $model_id_name_$seq_len'_'$pred_len \
--model $model_name \
--data $data_name \
--features M \
--seq_len $seq_len \
--pred_len $pred_len \
--enc_in 7 \
--n1 $n1 \
--n2 $n2 \
--dropout $fc_drop\
--revin 1\
--ch_ind 1\
--residual $residual\
--dconv $dconv \
--d_state $dstate\
--e_fact $e_fact\
--des 'Exp' \
--train_epochs 100\
--itr 1 --batch_size 512 --learning_rate 0.001 >logs/LongForecasting/$model_name'_'$model_id_name'_'$seq_len'_'$pred_len'_'$n1'_'$n2'_'$fc_drop'_'$rin'_'$residual'_'$dstate'_'$dconv'_'$e_fact.log
done
done
done
================================================
FILE: TimeMachine_supervised/scripts/TimeMachine/etth2.sh
================================================
if [ ! -d "./logs" ]; then
mkdir ./logs
fi
if [ ! -d "./logs/LongForecasting" ]; then
mkdir ./logs/LongForecasting
fi
if [ ! -d "./csv_results" ]; then
mkdir ./csv_results
fi
if [ ! -d "./results" ]; then
mkdir ./results
fi
if [ ! -d "./test_results" ]; then
mkdir ./test_results
fi
model_name=TimeMachine
root_path_name=../data/ETT-small
data_path_name=ETTh2.csv
model_id_name=ETTh2
data_name=ETTh2
one=96
two=192
three=336
four=720
residual=1
rin=1
fc_drop=0.7
dstate=256
random_seed=2024
dconv=2
for seq_len in 96
do
for pred_len in 96 192 336 720
do
for e_fact in 1
do
if [ $pred_len -eq $one ]
then
n1=128
n2=32
fi
if [ $pred_len -eq $two ]
then
n1=256
n2=32
fi
if [ $pred_len -eq $three ]
then
n1=512
n2=64
fi
if [ $pred_len -eq $four ]
then
n1=256
n2=128
fi
python -u run_longExp.py \
--random_seed $random_seed \
--is_training 1 \
--root_path $root_path_name \
--data_path $data_path_name \
--model_id $model_id_name_$seq_len'_'$pred_len \
--model $model_name \
--data $data_name \
--features M \
--seq_len $seq_len \
--pred_len $pred_len \
--enc_in 7 \
--n1 $n1 \
--n2 $n2 \
--dropout $fc_drop\
--revin 1\
--ch_ind 1\
--d_state $dstate\
--dconv $dconv \
--residual $residual\
--e_fact $e_fact\
--des 'Exp' \
--train_epochs 100\
--itr 1 --batch_size 512 --learning_rate 0.001 >logs/LongForecasting/$model_name'_'$model_id_name'_'$seq_len'_'$pred_len'_'$n1'_'$n2'_'$fc_drop'_'$rin'_'$residual'_'$dstate'_'$dconv'_'$e_fact.log
done
done
done
================================================
FILE: TimeMachine_supervised/scripts/TimeMachine/ettm1.sh
================================================
if [ ! -d "./logs" ]; then
mkdir ./logs
fi
if [ ! -d "./logs/LongForecasting" ]; then
mkdir ./logs/LongForecasting
fi
if [ ! -d "./csv_results" ]; then
mkdir ./csv_results
fi
if [ ! -d "./results" ]; then
mkdir ./results
fi
if [ ! -d "./test_results" ]; then
mkdir ./test_results
fi
model_name=TimeMachine
root_path_name=../data/ETT-small
data_path_name=ETTm1.csv
model_id_name=ETTm1
data_name=ETTm1
rin=1
random_seed=2024
one=96
two=192
three=336
four=720
residual=1
fc_drop=0.6
dstate=256
dconv=2
for seq_len in 96
do
for pred_len in 96 192 336 720
do
for e_fact in 1
do
if [ $pred_len -eq $one ]
then
n1=256
n2=64
fi
if [ $pred_len -eq $two ]
then
n1=512
n2=32
fi
if [ $pred_len -eq $three ]
then
n1=128
n2=16
fi
if [ $pred_len -eq $four ]
then
n1=512
n2=128
fi
python -u run_longExp.py \
--random_seed $random_seed \
--is_training 1 \
--root_path $root_path_name \
--data_path $data_path_name \
--model_id $model_id_name_$seq_len'_'$pred_len \
--model $model_name \
--data $data_name \
--features M \
--seq_len $seq_len \
--pred_len $pred_len \
--enc_in 7 \
--n1 $n1 \
--n2 $n2 \
--dropout $fc_drop\
--des 'Exp' \
--train_epochs 100\
--itr 1 --batch_size 1024 --learning_rate 0.001 >logs/LongForecasting/$model_name'_'$model_id_name'_'$seq_len'_'$pred_len'_'$n1'_'$n2'_'$fc_drop'_'$rin'_'$residual'_'$dstate'_'$dconv'_'$e_fact.log
done
done
done
================================================
FILE: TimeMachine_supervised/scripts/TimeMachine/ettm2.sh
================================================
if [ ! -d "./logs" ]; then
mkdir ./logs
fi
if [ ! -d "./logs/LongForecasting" ]; then
mkdir ./logs/LongForecasting
fi
if [ ! -d "./csv_results" ]; then
mkdir ./csv_results
fi
if [ ! -d "./results" ]; then
mkdir ./results
fi
if [ ! -d "./test_results" ]; then
mkdir ./test_results
fi
model_name=TimeMachine
root_path_name=../data/ETT-small
data_path_name=ETTm2.csv
model_id_name=ETTm2
data_name=ETTm2
rin=1
random_seed=2024
one=96
two=192
three=336
four=720
residual=1
fc_drop=0.6
dstate=256
dconv=2
for seq_len in 96
do
for pred_len in 96 192 336 720
do
for e_fact in 1
do
if [ $pred_len -eq $one ]
then
n1=256
n2=32
fi
if [ $pred_len -eq $two ]
then
n1=128
n2=64
fi
if [ $pred_len -eq $three ]
then
n1=256
n2=128
fi
if [ $pred_len -eq $four ]
then
n1=256
n2=128
fi
python -u run_longExp.py \
--random_seed $random_seed \
--is_training 1 \
--root_path $root_path_name \
--data_path $data_path_name \
--model_id $model_id_name_$seq_len'_'$pred_len \
--model $model_name \
--data $data_name \
--features M \
--seq_len $seq_len \
--pred_len $pred_len \
--enc_in 7 \
--n1 $n1 \
--n2 $n2 \
--dropout $fc_drop\
--des 'Exp' \
--train_epochs 100\
--itr 1 --batch_size 1024 --learning_rate 0.001 >logs/LongForecasting/$model_name'_'$model_id_name'_'$seq_len'_'$pred_len'_'$n1'_'$n2'_'$fc_drop'_'$rin'_'$residual'_'$dstate'_'$dconv'_'$e_fact.log
done
done
done
================================================
FILE: TimeMachine_supervised/scripts/TimeMachine/traffic.sh
================================================
if [ ! -d "./logs" ]; then
mkdir ./logs
fi
if [ ! -d "./logs/LongForecasting" ]; then
mkdir ./logs/LongForecasting
fi
if [ ! -d "./csv_results" ]; then
mkdir ./csv_results
fi
if [ ! -d "./results" ]; then
mkdir ./results
fi
if [ ! -d "./test_results" ]; then
mkdir ./test_results
fi
model_name=TimeMachine
root_path_name=../data/traffic
data_path_name=traffic.csv
model_id_name=traffic
data_name=custom
rin=0
random_seed=2024
one=96
two=192
three=336
four=720
residual=1
fc_drop=0.3
dstate=256
dconv=2
for seq_len in 96
do
for pred_len in 96 192 336 720
do
for e_fact in 1
do
if [ $pred_len -eq $one ]
then
n1=512
n2=16
fc_drop=0.3
fi
if [ $pred_len -eq $two ]
then
n1=512
n2=256
fc_drop=0.1
fi
if [ $pred_len -eq $three ]
then
n1=512
n2=256
fc_drop=0.1
fi
if [ $pred_len -eq $four ]
then
n1=512
n2=128
fc_drop=0.1
fi
python -u run_longExp.py \
--random_seed $random_seed \
--is_training 1 \
--root_path $root_path_name \
--data_path $data_path_name \
--model_id $model_id_name_$seq_len'_'$pred_len \
--model $model_name \
--data $data_name \
--features M \
--seq_len $seq_len \
--pred_len $pred_len \
--enc_in 862 \
--n1 $n1 \
--n2 $n2 \
--dropout $fc_drop \
--revin 0 \
--ch_ind 0 \
--residual $residual \
--dconv $dconv \
--d_state $dstate \
--e_fact $e_fact \
--des 'Exp' \
--lradj '5' \
--pct_start 0.2 \
--train_epochs 100 \
--itr 1 --batch_size 16 --learning_rate 0.001 >logs/LongForecasting/$model_name'_'$model_id_name'_'$seq_len'_'$pred_len'_'$n1'_'$n2'_'$fc_drop'_'$rin'_'$residual'_'$dstate'_'$dconv'_'$e_fact.log
done
done
done
================================================
FILE: TimeMachine_supervised/scripts/TimeMachine/weather.sh
================================================
if [ ! -d "./logs" ]; then
mkdir ./logs
fi
if [ ! -d "./logs/LongForecasting" ]; then
mkdir ./logs/LongForecasting
fi
if [ ! -d "./csv_results" ]; then
mkdir ./csv_results
fi
if [ ! -d "./results" ]; then
mkdir ./results
fi
if [ ! -d "./test_results" ]; then
mkdir ./test_results
fi
model_name=TimeMachine
root_path_name=../data/weather
data_path_name=weather.csv
model_id_name=weather
data_name=custom
rin=1
random_seed=2024
one=96
two=192
three=336
four=720
residual=1
fc_drop=0.1
dstate=256
dconv=2
for seq_len in 96
do
for pred_len in 96 192 336 720
do
for e_fact in 1
do
if [ $pred_len -eq $one ]
then
n1=128
n2=16
fc_drop=0.1
fi
if [ $pred_len -eq $two ]
then
n1=512
n2=128
fc_drop=0.5
fi
if [ $pred_len -eq $three ]
then
n1=128
n2=64
fc_drop=0.0
fi
if [ $pred_len -eq $four ]
then
n1=512
n2=256
fc_drop=0.0
fi
python -u run_longExp.py \
--random_seed $random_seed \
--is_training 1 \
--root_path $root_path_name \
--data_path $data_path_name \
--model_id $model_id_name_$seq_len'_'$pred_len \
--model $model_name \
--data $data_name \
--features M \
--seq_len $seq_len \
--pred_len $pred_len \
--enc_in 21 \
--n1 $n1 \
--n2 $n2 \
--dropout $fc_drop\
--revin 1\
--ch_ind 1\
--residual $residual\
--dconv $dconv \
--d_state $dstate\
--e_fact $e_fact\
--des 'Exp' \
--lradj 'constant'\
--pct_start 0.2\
--itr 1 --batch_size 512 --learning_rate 0.001 >logs/LongForecasting/$model_name'_'$model_id_name'_'$seq_len'_'$pred_len'_'$n1'_'$n2'_'$fc_drop'_'$rin'_'$residual'_'$dstate'_'$dconv'_'$e_fact.log
done
done
done
================================================
FILE: TimeMachine_supervised/utils/masking.py
================================================
import torch
class TriangularCausalMask():
def __init__(self, B, L, device="cpu"):
mask_shape = [B, 1, L, L]
with torch.no_grad():
self._mask = torch.triu(torch.ones(mask_shape, dtype=torch.bool), diagonal=1).to(device)
@property
def mask(self):
return self._mask
class ProbMask():
def __init__(self, B, H, L, index, scores, device="cpu"):
_mask = torch.ones(L, scores.shape[-1], dtype=torch.bool).to(device).triu(1)
_mask_ex = _mask[None, None, :].expand(B, H, L, scores.shape[-1])
indicator = _mask_ex[torch.arange(B)[:, None, None],
torch.arange(H)[None, :, None],
index, :].to(device)
self._mask = indicator.view(scores.shape).to(device)
@property
def mask(self):
return self._mask
================================================
FILE: TimeMachine_supervised/utils/metrics.py
================================================
import numpy as np
def RSE(pred, true):
return np.sqrt(np.sum((true - pred) ** 2)) / np.sqrt(np.sum((true - true.mean()) ** 2))
def CORR(pred, true):
u = ((true - true.mean(0)) * (pred - pred.mean(0))).sum(0)
d = np.sqrt(((true - true.mean(0)) ** 2 * (pred - pred.mean(0)) ** 2).sum(0))
d += 1e-12
return 0.01*(u / d).mean(-1)
def MAE(pred, true):
return np.mean(np.abs(pred - true))
def MSE(pred, true):
return np.mean((pred - true) ** 2)
def RMSE(pred, true):
return np.sqrt(MSE(pred, true))
def MAPE(pred, true):
return np.mean(np.abs((pred - true) / true))
def MSPE(pred, true):
return np.mean(np.square((pred - true) / true))
def metric(pred, true):
mae = MAE(pred, true)
mse = MSE(pred, true)
rmse = RMSE(pred, true)
mape = MAPE(pred, true)
mspe = MSPE(pred, true)
rse = RSE(pred, true)
corr = CORR(pred, true)
return mae, mse, rmse, mape, mspe, rse, corr
================================================
FILE: TimeMachine_supervised/utils/timefeatures.py
================================================
from typing import List
import numpy as np
import pandas as pd
from pandas.tseries import offsets
from pandas.tseries.frequencies import to_offset
class TimeFeature:
def __init__(self):
pass
def __call__(self, index: pd.DatetimeIndex) -> np.ndarray:
pass
def __repr__(self):
return self.__class__.__name__ + "()"
class SecondOfMinute(TimeFeature):
"""Minute of hour encoded as value between [-0.5, 0.5]"""
def __call__(self, index: pd.DatetimeIndex) -> np.ndarray:
return index.second / 59.0 - 0.5
class MinuteOfHour(TimeFeature):
"""Minute of hour encoded as value between [-0.5, 0.5]"""
def __call__(self, index: pd.DatetimeIndex) -> np.ndarray:
return index.minute / 59.0 - 0.5
class HourOfDay(TimeFeature):
"""Hour of day encoded as value between [-0.5, 0.5]"""
def __call__(self, index: pd.DatetimeIndex) -> np.ndarray:
return index.hour / 23.0 - 0.5
class DayOfWeek(TimeFeature):
"""Hour of day encoded as value between [-0.5, 0.5]"""
def __call__(self, index: pd.DatetimeIndex) -> np.ndarray:
return index.dayofweek / 6.0 - 0.5
class DayOfMonth(TimeFeature):
"""Day of month encoded as value between [-0.5, 0.5]"""
def __call__(self, index: pd.DatetimeIndex) -> np.ndarray:
return (index.day - 1) / 30.0 - 0.5
class DayOfYear(TimeFeature):
"""Day of year encoded as value between [-0.5, 0.5]"""
def __call__(self, index: pd.DatetimeIndex) -> np.ndarray:
return (index.dayofyear - 1) / 365.0 - 0.5
class MonthOfYear(TimeFeature):
"""Month of year encoded as value between [-0.5, 0.5]"""
def __call__(self, index: pd.DatetimeIndex) -> np.ndarray:
return (index.month - 1) / 11.0 - 0.5
class WeekOfYear(TimeFeature):
"""Week of year encoded as value between [-0.5, 0.5]"""
def __call__(self, index: pd.DatetimeIndex) -> np.ndarray:
return (index.isocalendar().week - 1) / 52.0 - 0.5
def time_features_from_frequency_str(freq_str: str) -> List[TimeFeature]:
"""
Returns a list of time features that will be appropriate for the given frequency string.
Parameters
----------
freq_str
Frequency string of the form [multiple][granularity] such as "12H", "5min", "1D" etc.
"""
features_by_offsets = {
offsets.YearEnd: [],
offsets.QuarterEnd: [MonthOfYear],
offsets.MonthEnd: [MonthOfYear],
offsets.Week: [DayOfMonth, WeekOfYear],
offsets.Day: [DayOfWeek, DayOfMonth, DayOfYear],
offsets.BusinessDay: [DayOfWeek, DayOfMonth, DayOfYear],
offsets.Hour: [HourOfDay, DayOfWeek, DayOfMonth, DayOfYear],
offsets.Minute: [
MinuteOfHour,
HourOfDay,
DayOfWeek,
DayOfMonth,
DayOfYear,
],
offsets.Second: [
SecondOfMinute,
MinuteOfHour,
HourOfDay,
DayOfWeek,
DayOfMonth,
DayOfYear,
],
}
offset = to_offset(freq_str)
for offset_type, feature_classes in features_by_offsets.items():
if isinstance(offset, offset_type):
return [cls() for cls in feature_classes]
supported_freq_msg = f"""
Unsupported frequency {freq_str}
The following frequencies are supported:
Y - yearly
alias: A
M - monthly
W - weekly
D - daily
B - business days
H - hourly
T - minutely
alias: min
S - secondly
"""
raise RuntimeError(supported_freq_msg)
def time_features(dates, freq='h'):
return np.vstack([feat(dates) for feat in time_features_from_frequency_str(freq)])
================================================
FILE: TimeMachine_supervised/utils/tools.py
================================================
import numpy as np
import torch
import matplotlib.pyplot as plt
import time
plt.switch_backend('agg')
def adjust_learning_rate(optimizer, scheduler, epoch, args, printout=True):
# lr = args.learning_rate * (0.2 ** (epoch // 2))
if args.lradj == 'type1':
lr_adjust = {epoch: args.learning_rate * (0.5 ** ((epoch - 1) // 1))}
elif args.lradj == 'type2':
lr_adjust = {
2: 5e-5, 4: 1e-5, 6: 5e-6, 8: 1e-6,
10: 5e-7, 15: 1e-7, 20: 5e-8
}
elif args.lradj == 'type3':
lr_adjust = {epoch: args.learning_rate if epoch < 3 else args.learning_rate * (0.9 ** ((epoch - 3) // 1))}
elif args.lradj == 'constant':
lr_adjust = {epoch: args.learning_rate}
elif args.lradj == '3':
lr_adjust = {epoch: args.learning_rate if epoch < 10 else args.learning_rate*0.1}
elif args.lradj == '4':
lr_adjust = {epoch: args.learning_rate if epoch < 15 else args.learning_rate*0.1}
elif args.lradj == '5':
lr_adjust = {epoch: args.learning_rate if epoch < 25 else args.learning_rate*0.1}
elif args.lradj == '6':
lr_adjust = {epoch: args.learning_rate if epoch < 5 else args.learning_rate*0.1}
elif args.lradj == 'TST':
lr_adjust = {epoch: scheduler.get_last_lr()[0]}
if epoch in lr_adjust.keys():
lr = lr_adjust[epoch]
for param_group in optimizer.param_groups:
param_group['lr'] = lr
if printout: print('Updating learning rate to {}'.format(lr))
class EarlyStopping:
def __init__(self, patience=7, verbose=False, delta=0):
self.patience = patience
self.verbose = verbose
self.counter = 0
self.best_score = None
self.early_stop = False
self.val_loss_min = np.Inf
self.delta = delta
def __call__(self, val_loss, model, path):
score = -val_loss
if self.best_score is None:
self.best_score = score
self.save_checkpoint(val_loss, model, path)
elif score < self.best_score + self.delta:
self.counter += 1
print(f'EarlyStopping counter: {self.counter} out of {self.patience}')
if self.counter >= self.patience:
self.early_stop = True
else:
self.best_score = score
self.save_checkpoint(val_loss, model, path)
self.counter = 0
def save_checkpoint(self, val_loss, model, path):
if self.verbose:
print(f'Validation loss decreased ({self.val_loss_min:.6f} --> {val_loss:.6f}). Saving model ...')
torch.save(model.state_dict(), path + '/' + 'checkpoint.pth')
self.val_loss_min = val_loss
class dotdict(dict):
"""dot.notation access to dictionary attributes"""
__getattr__ = dict.get
__setattr__ = dict.__setitem__
__delattr__ = dict.__delitem__
class StandardScaler():
def __init__(self, mean, std):
self.mean = mean
self.std = std
def transform(self, data):
return (data - self.mean) / self.std
def inverse_transform(self, data):
return (data * self.std) + self.mean
def visual(true, preds=None, name='./pic/test.pdf'):
"""
Results visualization
"""
plt.figure()
plt.plot(true, label='GroundTruth', linewidth=2)
if preds is not None:
plt.plot(preds, label='Prediction', linewidth=2)
plt.legend()
plt.savefig(name, bbox_inches='tight')
def test_params_flop(model,x_shape):
"""
If you want to thest former's flop, you need to give default value to inputs in model.forward(), the following code can only pass one argument to forward()
"""
model_params = 0
for parameter in model.parameters():
model_params += parameter.numel()
print('INFO: Trainable parameter count: {:.2f}M'.format(model_params / 1000000.0))
from ptflops import get_model_complexity_info
with torch.cuda.device(0):
macs, params = get_model_complexity_info(model.cuda(), x_shape, as_strings=True, print_per_layer_stat=True)
# print('Flops:' + flops)
# print('Params:' + params)
print('{:<30} {:<8}'.format('Computational complexity: ', macs))
print('{:<30} {:<8}'.format('Number of parameters: ', params))
gitextract_wg48y8vq/
├── .gitignore
├── LICENSE
├── README.md
└── TimeMachine_supervised/
├── RevIN/
│ └── RevIN.py
├── data_provider/
│ ├── data_factory.py
│ └── data_loader.py
├── exp/
│ ├── exp_basic.py
│ └── exp_main.py
├── models/
│ └── TimeMachine.py
├── requirements.txt
├── run_longExp.py
├── scripts/
│ └── TimeMachine/
│ ├── electricity.sh
│ ├── etth1.sh
│ ├── etth2.sh
│ ├── ettm1.sh
│ ├── ettm2.sh
│ ├── traffic.sh
│ └── weather.sh
└── utils/
├── masking.py
├── metrics.py
├── timefeatures.py
└── tools.py
SYMBOL INDEX (101 symbols across 10 files)
FILE: TimeMachine_supervised/RevIN/RevIN.py
class RevIN (line 4) | class RevIN(nn.Module):
method __init__ (line 5) | def __init__(self, num_features: int, eps=1e-5, affine=True):
method forward (line 18) | def forward(self, x, mode:str):
method _init_params (line 27) | def _init_params(self):
method _get_statistics (line 32) | def _get_statistics(self, x):
method _normalize (line 37) | def _normalize(self, x):
method _denormalize (line 45) | def _denormalize(self, x):
FILE: TimeMachine_supervised/data_provider/data_factory.py
function data_provider (line 13) | def data_provider(args, flag):
FILE: TimeMachine_supervised/data_provider/data_loader.py
class Dataset_ETT_hour (line 14) | class Dataset_ETT_hour(Dataset):
method __init__ (line 15) | def __init__(self, root_path, flag='train', size=None,
method __read_data__ (line 43) | def __read_data__(self):
method __getitem__ (line 82) | def __getitem__(self, index):
method __len__ (line 95) | def __len__(self):
method inverse_transform (line 98) | def inverse_transform(self, data):
class Dataset_ETT_minute (line 102) | class Dataset_ETT_minute(Dataset):
method __init__ (line 103) | def __init__(self, root_path, flag='train', size=None,
method __read_data__ (line 131) | def __read_data__(self):
method __getitem__ (line 172) | def __getitem__(self, index):
method __len__ (line 185) | def __len__(self):
method inverse_transform (line 188) | def inverse_transform(self, data):
class Dataset_Custom (line 192) | class Dataset_Custom(Dataset):
method __init__ (line 193) | def __init__(self, root_path, flag='train', size=None,
method __read_data__ (line 221) | def __read_data__(self):
method __getitem__ (line 273) | def __getitem__(self, index):
method __len__ (line 286) | def __len__(self):
method inverse_transform (line 289) | def inverse_transform(self, data):
class Dataset_Pred (line 293) | class Dataset_Pred(Dataset):
method __init__ (line 294) | def __init__(self, root_path, flag='pred', size=None,
method __read_data__ (line 321) | def __read_data__(self):
method __getitem__ (line 376) | def __getitem__(self, index):
method __len__ (line 392) | def __len__(self):
method inverse_transform (line 395) | def inverse_transform(self, data):
FILE: TimeMachine_supervised/exp/exp_basic.py
class Exp_Basic (line 6) | class Exp_Basic(object):
method __init__ (line 7) | def __init__(self, args):
method _build_model (line 12) | def _build_model(self):
method _acquire_device (line 16) | def _acquire_device(self):
method _get_data (line 27) | def _get_data(self):
method vali (line 30) | def vali(self):
method train (line 33) | def train(self):
method test (line 36) | def test(self):
FILE: TimeMachine_supervised/exp/exp_main.py
class Exp_Main (line 22) | class Exp_Main(Exp_Basic):
method __init__ (line 23) | def __init__(self, args):
method _build_model (line 26) | def _build_model(self):
method _get_data (line 36) | def _get_data(self, flag):
method _select_optimizer (line 40) | def _select_optimizer(self):
method _select_criterion (line 44) | def _select_criterion(self):
method vali (line 48) | def vali(self, vali_data, vali_loader, criterion):
method train (line 85) | def train(self, setting):
method test (line 196) | def test(self, setting, test=0):
method predict (line 315) | def predict(self, setting, load=False):
FILE: TimeMachine_supervised/models/TimeMachine.py
class Model (line 4) | class Model(torch.nn.Module):
method __init__ (line 5) | def __init__(self,configs):
method forward (line 36) | def forward(self, x):
FILE: TimeMachine_supervised/utils/masking.py
class TriangularCausalMask (line 4) | class TriangularCausalMask():
method __init__ (line 5) | def __init__(self, B, L, device="cpu"):
method mask (line 11) | def mask(self):
class ProbMask (line 15) | class ProbMask():
method __init__ (line 16) | def __init__(self, B, H, L, index, scores, device="cpu"):
method mask (line 25) | def mask(self):
FILE: TimeMachine_supervised/utils/metrics.py
function RSE (line 4) | def RSE(pred, true):
function CORR (line 8) | def CORR(pred, true):
function MAE (line 15) | def MAE(pred, true):
function MSE (line 19) | def MSE(pred, true):
function RMSE (line 23) | def RMSE(pred, true):
function MAPE (line 27) | def MAPE(pred, true):
function MSPE (line 31) | def MSPE(pred, true):
function metric (line 35) | def metric(pred, true):
FILE: TimeMachine_supervised/utils/timefeatures.py
class TimeFeature (line 9) | class TimeFeature:
method __init__ (line 10) | def __init__(self):
method __call__ (line 13) | def __call__(self, index: pd.DatetimeIndex) -> np.ndarray:
method __repr__ (line 16) | def __repr__(self):
class SecondOfMinute (line 20) | class SecondOfMinute(TimeFeature):
method __call__ (line 23) | def __call__(self, index: pd.DatetimeIndex) -> np.ndarray:
class MinuteOfHour (line 27) | class MinuteOfHour(TimeFeature):
method __call__ (line 30) | def __call__(self, index: pd.DatetimeIndex) -> np.ndarray:
class HourOfDay (line 34) | class HourOfDay(TimeFeature):
method __call__ (line 37) | def __call__(self, index: pd.DatetimeIndex) -> np.ndarray:
class DayOfWeek (line 41) | class DayOfWeek(TimeFeature):
method __call__ (line 44) | def __call__(self, index: pd.DatetimeIndex) -> np.ndarray:
class DayOfMonth (line 48) | class DayOfMonth(TimeFeature):
method __call__ (line 51) | def __call__(self, index: pd.DatetimeIndex) -> np.ndarray:
class DayOfYear (line 55) | class DayOfYear(TimeFeature):
method __call__ (line 58) | def __call__(self, index: pd.DatetimeIndex) -> np.ndarray:
class MonthOfYear (line 62) | class MonthOfYear(TimeFeature):
method __call__ (line 65) | def __call__(self, index: pd.DatetimeIndex) -> np.ndarray:
class WeekOfYear (line 69) | class WeekOfYear(TimeFeature):
method __call__ (line 72) | def __call__(self, index: pd.DatetimeIndex) -> np.ndarray:
function time_features_from_frequency_str (line 76) | def time_features_from_frequency_str(freq_str: str) -> List[TimeFeature]:
function time_features (line 133) | def time_features(dates, freq='h'):
FILE: TimeMachine_supervised/utils/tools.py
function adjust_learning_rate (line 9) | def adjust_learning_rate(optimizer, scheduler, epoch, args, printout=True):
class EarlyStopping (line 40) | class EarlyStopping:
method __init__ (line 41) | def __init__(self, patience=7, verbose=False, delta=0):
method __call__ (line 50) | def __call__(self, val_loss, model, path):
method save_checkpoint (line 65) | def save_checkpoint(self, val_loss, model, path):
class dotdict (line 72) | class dotdict(dict):
class StandardScaler (line 79) | class StandardScaler():
method __init__ (line 80) | def __init__(self, mean, std):
method transform (line 84) | def transform(self, data):
method inverse_transform (line 87) | def inverse_transform(self, data):
function visual (line 91) | def visual(true, preds=None, name='./pic/test.pdf'):
function test_params_flop (line 102) | def test_params_flop(model,x_shape):
Condensed preview — 22 files, each showing path, character count, and a content snippet. Download the .json file or copy for the full structured content (87K chars).
[
{
"path": ".gitignore",
"chars": 465,
"preview": "data/\nTimeMachine_supervised/exp/__pycache__/\nTimeMachine_supervised/data_provider/__pycache__/\nTimeMachine_supervised/m"
},
{
"path": "LICENSE",
"chars": 11357,
"preview": " Apache License\n Version 2.0, January 2004\n "
},
{
"path": "README.md",
"chars": 2221,
"preview": "# <center>TimeMachine</center>\n\n\n### Welcome to the official repository of: [TimeMach"
},
{
"path": "TimeMachine_supervised/RevIN/RevIN.py",
"chars": 1694,
"preview": "import torch\nimport torch.nn as nn\n\nclass RevIN(nn.Module):\n def __init__(self, num_features: int, eps=1e-5, affine=T"
},
{
"path": "TimeMachine_supervised/data_provider/data_factory.py",
"chars": 1372,
"preview": "from data_provider.data_loader import Dataset_ETT_hour, Dataset_ETT_minute, Dataset_Custom, Dataset_Pred\nfrom torch.util"
},
{
"path": "TimeMachine_supervised/data_provider/data_loader.py",
"chars": 14720,
"preview": "import os\nimport numpy as np\nimport pandas as pd\nimport os\nimport torch\nfrom torch.utils.data import Dataset, DataLoader"
},
{
"path": "TimeMachine_supervised/exp/exp_basic.py",
"chars": 885,
"preview": "import os\nimport torch\nimport numpy as np\n\n\nclass Exp_Basic(object):\n def __init__(self, args):\n self.args = a"
},
{
"path": "TimeMachine_supervised/exp/exp_main.py",
"chars": 14947,
"preview": "from data_provider.data_factory import data_provider\nfrom exp.exp_basic import Exp_Basic\nfrom models import TimeMachine\n"
},
{
"path": "TimeMachine_supervised/models/TimeMachine.py",
"chars": 3338,
"preview": "import torch\nfrom mamba_ssm import Mamba\nfrom RevIN.RevIN import RevIN\nclass Model(torch.nn.Module):\n def __init__(se"
},
{
"path": "TimeMachine_supervised/requirements.txt",
"chars": 73,
"preview": "numpy\nmatplotlib\npandas\nscikit-learn\ntorch\nmamba-ssm\ncausal-conv1d>=1.2.0"
},
{
"path": "TimeMachine_supervised/run_longExp.py",
"chars": 7468,
"preview": "import argparse\nimport os\nimport torch\nfrom exp.exp_main import Exp_Main\nimport random\nimport numpy as np\n\nif __name__ ="
},
{
"path": "TimeMachine_supervised/scripts/TimeMachine/electricity.sh",
"chars": 2273,
"preview": "if [ ! -d \"./logs\" ]; then\n mkdir ./logs\nfi\n\nif [ ! -d \"./logs/LongForecasting\" ]; then\n mkdir ./logs/LongForecast"
},
{
"path": "TimeMachine_supervised/scripts/TimeMachine/etth1.sh",
"chars": 2088,
"preview": "if [ ! -d \"./logs\" ]; then\n mkdir ./logs\nfi\n\nif [ ! -d \"./logs/LongForecasting\" ]; then\n mkdir ./logs/LongForecast"
},
{
"path": "TimeMachine_supervised/scripts/TimeMachine/etth2.sh",
"chars": 2087,
"preview": "if [ ! -d \"./logs\" ]; then\n mkdir ./logs\nfi\n\nif [ ! -d \"./logs/LongForecasting\" ]; then\n mkdir ./logs/LongForecast"
},
{
"path": "TimeMachine_supervised/scripts/TimeMachine/ettm1.sh",
"chars": 1958,
"preview": "if [ ! -d \"./logs\" ]; then\n mkdir ./logs\nfi\n\nif [ ! -d \"./logs/LongForecasting\" ]; then\n mkdir ./logs/LongForecast"
},
{
"path": "TimeMachine_supervised/scripts/TimeMachine/ettm2.sh",
"chars": 1907,
"preview": "if [ ! -d \"./logs\" ]; then\n mkdir ./logs\nfi\n\nif [ ! -d \"./logs/LongForecasting\" ]; then\n mkdir ./logs/LongForecast"
},
{
"path": "TimeMachine_supervised/scripts/TimeMachine/traffic.sh",
"chars": 2270,
"preview": "if [ ! -d \"./logs\" ]; then\n mkdir ./logs\nfi\n\nif [ ! -d \"./logs/LongForecasting\" ]; then\n mkdir ./logs/LongForecast"
},
{
"path": "TimeMachine_supervised/scripts/TimeMachine/weather.sh",
"chars": 2234,
"preview": "if [ ! -d \"./logs\" ]; then\n mkdir ./logs\nfi\n\nif [ ! -d \"./logs/LongForecasting\" ]; then\n mkdir ./logs/LongForecast"
},
{
"path": "TimeMachine_supervised/utils/masking.py",
"chars": 832,
"preview": "import torch\n\n\nclass TriangularCausalMask():\n def __init__(self, B, L, device=\"cpu\"):\n mask_shape = [B, 1, L, "
},
{
"path": "TimeMachine_supervised/utils/metrics.py",
"chars": 951,
"preview": "import numpy as np\n\n\ndef RSE(pred, true):\n return np.sqrt(np.sum((true - pred) ** 2)) / np.sqrt(np.sum((true - true.m"
},
{
"path": "TimeMachine_supervised/utils/timefeatures.py",
"chars": 3743,
"preview": "from typing import List\n\nimport numpy as np\nimport pandas as pd\nfrom pandas.tseries import offsets\nfrom pandas.tseries.f"
},
{
"path": "TimeMachine_supervised/utils/tools.py",
"chars": 4273,
"preview": "import numpy as np\nimport torch\nimport matplotlib.pyplot as plt\nimport time\n\nplt.switch_backend('agg')\n\n\ndef adjust_lear"
}
]
About this extraction
This page contains the full source code of the Atik-Ahamed/TimeMachine GitHub repository, extracted and formatted as plain text for AI agents and large language models (LLMs). The extraction includes 22 files (81.2 KB), approximately 21.3k tokens, and a symbol index with 101 extracted functions, classes, methods, constants, and types. Use this with OpenClaw, Claude, ChatGPT, Cursor, Windsurf, or any other AI tool that accepts text input. You can copy the full output to your clipboard or download it as a .txt file.
Extracted by GitExtract — free GitHub repo to text converter for AI. Built by Nikandr Surkov.