Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

【hydra No.11】for hpinns model #582

Merged
merged 7 commits into from
Oct 25, 2023
Merged

【hydra No.11】for hpinns model #582

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
95ff433
Add hpinns hydra
co63oc Oct 20, 2023
75608d9
Fix
co63oc Oct 20, 2023
fe16eb1
Fix
co63oc Oct 20, 2023
1c8ae38
Fix
co63oc Oct 20, 2023
05e3d23
Merge branch 'develop' into hpinns
co63oc Oct 20, 2023
db3c978
Fix
co63oc Oct 24, 2023
4de81e2
Fix
co63oc Oct 25, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
82 changes: 50 additions & 32 deletions docs/zh/examples/hpinns.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,30 @@

<a href="https://aistudio.baidu.com/aistudio/projectdetail/6390502" class="md-button md-button--primary" style>AI Studio快速体验</a>

=== "模型训练命令"

``` sh
# linux
wget -P ./datasets/ https://paddle-org.bj.bcebos.com/paddlescience/datasets/hPINNs/hpinns_holo_train.mat
wget -P ./datasets/ https://paddle-org.bj.bcebos.com/paddlescience/datasets/hPINNs/hpinns_holo_valid.mat
# windows
# curl https://paddle-org.bj.bcebos.com/paddlescience/datasets/hPINNs/hpinns_holo_train.mat --output ./datasets/hpinns_holo_train.mat
# curl https://paddle-org.bj.bcebos.com/paddlescience/datasets/hPINNs/hpinns_holo_valid.mat --output ./datasets/hpinns_holo_valid.mat
python holography.py
```

=== "模型评估命令"

``` sh
# linux
wget -P ./datasets/ https://paddle-org.bj.bcebos.com/paddlescience/datasets/hPINNs/hpinns_holo_train.mat
wget -P ./datasets/ https://paddle-org.bj.bcebos.com/paddlescience/datasets/hPINNs/hpinns_holo_valid.mat
# windows
# curl https://paddle-org.bj.bcebos.com/paddlescience/datasets/hPINNs/hpinns_holo_train.mat --output ./datasets/hpinns_holo_train.mat
# curl https://paddle-org.bj.bcebos.com/paddlescience/datasets/hPINNs/hpinns_holo_valid.mat --output ./datasets/hpinns_holo_valid.mat
python holography.py mode=eval EVAL.pretrained_model_path=https://paddle-org.bj.bcebos.com/paddlescience/models/hpinns/hpinns_pretrained.pdparams
```

## 1. 背景简介

求解偏微分方程(PDE) 是一类基础的物理问题，在过去几十年里，以有限差分(FDM)、有限体积(FVM)、有限元(FEM)为代表的多种偏微分方程组数值解法趋于成熟。随着人工智能技术的高速发展，利用深度学习求解偏微分方程成为新的研究趋势。PINNs(Physics-informed neural networks) 是一种加入物理约束的深度学习网络，因此与纯数据驱动的神经网络学习相比，PINNs 可以用更少的数据样本学习到更具泛化能力的模型，其应用范围包括但不限于流体力学、热传导、电磁场、量子力学等领域。
Expand Down Expand Up @@ -84,9 +108,9 @@ $$

上式中 $f_1,f_2,f_3$ 分别为一个 MLP 模型，三者共同构成了一个 Model List，用 PaddleScience 代码表示如下

``` py linenums="43"
``` py linenums="42"
--8<--
examples/hpinns/holography.py:43:51
examples/hpinns/holography.py:42:44
--8<--
```

Expand All @@ -105,9 +129,9 @@ examples/hpinns/functions.py:49:92

需要对每个 MLP 模型分别注册相应的 transform ，然后将 3 个 MLP 模型组成 Model List

``` py linenums="59"
``` py linenums="50"
--8<--
examples/hpinns/holography.py:59:68
examples/hpinns/holography.py:50:59
--8<--
```

Expand All @@ -117,15 +141,15 @@ examples/hpinns/holography.py:59:68

我们需要指定问题相关的参数，如通过 `train_mode` 参数指定应用增强的拉格朗日方法的硬约束进行训练

``` py linenums="31"
``` py linenums="35"
--8<--
examples/hpinns/holography.py:31:41
examples/hpinns/holography.py:35:40
--8<--
```

``` py linenums="53"
``` py linenums="46"
--8<--
examples/hpinns/holography.py:53:57
examples/hpinns/holography.py:46:48
--8<--
```

Expand All @@ -141,31 +165,25 @@ $\mu_k = \beta \mu_{k-1}$, $\lambda_k = \beta \lambda_{k-1}$

同时需要指定训练轮数和学习率等超参数

``` py linenums="70"
--8<--
examples/hpinns/holography.py:70:72
--8<--
```

``` py linenums="212"
``` py linenums="53"
--8<--
examples/hpinns/holography.py:212:214
examples/hpinns/conf/hpinns.yaml:53:61
--8<--
```

### 3.5 优化器构建

训练分为两个阶段，先使用 Adam 优化器进行大致训练，再使用 LBFGS 优化器逼近最优点，因此需要两个优化器，这也对应了上一部分超参数中的两种 `EPOCHS` 值

``` py linenums="74"
``` py linenums="62"
--8<--
examples/hpinns/holography.py:74:75
examples/hpinns/holography.py:62:64
--8<--
```

``` py linenums="216"
``` py linenums="203"
--8<--
examples/hpinns/holography.py:216:219
examples/hpinns/holography.py:203:205
--8<--
```

Expand All @@ -175,19 +193,19 @@ examples/hpinns/holography.py:216:219

虽然我们不是以监督学习方式进行训练，但此处仍然可以采用监督约束 `SupervisedConstraint`，在定义约束之前，需要给监督约束指定文件路径等数据读取配置，因为数据集中没有标签数据，因此在数据读取时我们需要使用训练数据充当标签数据，并注意在之后不要使用这部分“假的”标签数据。

``` py linenums="113"
``` py linenums="102"
--8<--
examples/hpinns/holography.py:113:118
examples/hpinns/holography.py:102:107
--8<--
```

如上，所有输出的标签都会读取输入 `x` 的值。

下面是约束等具体内容，要注意上述提到的给定“假的”标签数据：

``` py linenums="77"
``` py linenums="66"
--8<--
examples/hpinns/holography.py:77:138
examples/hpinns/holography.py:66:127
--8<--
```

Expand All @@ -207,19 +225,19 @@ examples/hpinns/holography.py:77:138

在约束构建完毕之后，以我们刚才的命名为关键字，封装到一个字典中，方便后续访问。

``` py linenums="139"
``` py linenums="128"
--8<--
examples/hpinns/holography.py:139:142
examples/hpinns/holography.py:128:131
--8<--
```

### 3.7 评估器构建

与约束同理，虽然本问题使用无监督学习，但仍可以使用 `ppsci.validate.SupervisedValidator` 构建评估器。

``` py linenums="144"
``` py linenums="133"
--8<--
examples/hpinns/holography.py:144:192
examples/hpinns/holography.py:133:181
--8<--
```

Expand Down Expand Up @@ -249,9 +267,9 @@ examples/hpinns/functions.py:320:336

完成上述设置之后，只需要将上述实例化的对象按顺序传递给 `ppsci.solver.Solver`，然后启动训练、评估。

``` py linenums="194"
``` py linenums="183"
--8<--
examples/hpinns/holography.py:194:210
examples/hpinns/holography.py:183:200
--8<--
```

Expand All @@ -261,9 +279,9 @@ examples/hpinns/holography.py:194:210

PaddleScience 中提供了可视化器，但由于本问题图片数量较多且较为复杂，代码中自定义了可视化函数，调用自定义函数即可实现可视化

``` py linenums="289"
``` py linenums="279"
--8<--
examples/hpinns/holography.py:289:
examples/hpinns/holography.py:279:292
--8<--
```

Expand Down
67 changes: 67 additions & 0 deletions examples/hpinns/conf/hpinns.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
hydra:
run:
# dynamic output directory according to running time and override name
dir: outputs_hpinns/${now:%Y-%m-%d}/${now:%H-%M-%S}/${hydra.job.override_dirname}
job:
name: ${mode} # name of logfile
chdir: false # keep current working direcotry unchanged
config:
override_dirname:
exclude_keys:
- TRAIN.checkpoint_path
- TRAIN.pretrained_model_path
- EVAL.pretrained_model_path
- mode
- output_dir
- log_freq
sweep:
# output directory for multirun
dir: ${hydra.run.dir}
subdir: ./

# general settings
mode: train # running mode: train/eval
seed: 42
output_dir: ${hydra:run.dir}
DATASET_PATH: ./datasets/hpinns_holo_train.mat
DATASET_PATH_VALID: ./datasets/hpinns_holo_valid.mat

# set working condition
TRAIN_MODE: aug_lag # "soft", "penalty", "aug_lag"
TRAIN_K: 9

# model settings
MODEL:
re_net:
input_keys: ['x_cos_1', 'x_sin_1', 'x_cos_2', 'x_sin_2', 'x_cos_3', 'x_sin_3', 'x_cos_4', 'x_sin_4', 'x_cos_5', 'x_sin_5', 'x_cos_6', 'x_sin_6', 'y', 'y_cos_1', 'y_sin_1']
output_keys: ["e_re"]
num_layers: 4
hidden_size: 48
activation: "tanh"
im_net:
input_keys: ${MODEL.re_net.input_keys}
output_keys: ["e_im"]
num_layers: 4
hidden_size: 48
activation: "tanh"
eps_net:
input_keys: ${MODEL.re_net.input_keys}
output_keys: ["eps"]
num_layers: 4
hidden_size: 48
activation: "tanh"

# training settings
TRAIN:
epochs: 20000
iters_per_epoch: 1
eval_during_train: false
learning_rate: 0.001
max_iter: 15000
epochs_lbfgs: 1
pretrained_model_path: null
checkpoint_path: null

# evaluation settings
EVAL:
pretrained_model_path: null
Loading