Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

【hydra No.13、14、16】lorenz/rossler/cylinder2d_unsteady_transformer_physx #602

Merged
merged 9 commits into from
Nov 2, 2023
Merged

【hydra No.13、14、16】lorenz/rossler/cylinder2d_unsteady_transformer_physx #602

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
e11c7b8
Convert hydra example
co63oc Oct 27, 2023
539e66c
Fix
co63oc Oct 27, 2023
e65496f
Merge branch 'develop' into cy2d
co63oc Oct 27, 2023
f0a6ecd
Fix
co63oc Oct 27, 2023
3467308
Fix
co63oc Oct 28, 2023
d5aaf0c
Fix
co63oc Oct 28, 2023
d172819
Merge branch 'develop' into cy2d
co63oc Oct 30, 2023
a41c60f
Fix
co63oc Oct 30, 2023
728a7a0
Fix
co63oc Oct 31, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
98 changes: 62 additions & 36 deletions docs/zh/examples/cylinder2d_unsteady_transformer_physx.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,32 @@

<a href="https://aistudio.baidu.com/aistudio/projectdetail/6178818?sUid=455441&shared=1&ts=1684397945680" class="md-button md-button--primary" style>AI Studio快速体验</a>

=== "模型训练命令"

``` sh
# linux
wget https://paddle-org.bj.bcebos.com/paddlescience/datasets/transformer_physx/cylinder_training.hdf5
wget https://paddle-org.bj.bcebos.com/paddlescience/datasets/transformer_physx/cylinder_valid.hdf5
# windows
# curl https://paddle-org.bj.bcebos.com/paddlescience/datasets/transformer_physx/cylinder_training.hdf5 --output cylinder_training.hdf5
# curl https://paddle-org.bj.bcebos.com/paddlescience/datasets/transformer_physx/cylinder_valid.hdf5 --output cylinder_valid.hdf5
python train_enn.py
python train_transformer.py
```

=== "模型评估命令"

``` sh
# linux
wget https://paddle-org.bj.bcebos.com/paddlescience/datasets/transformer_physx/cylinder_training.hdf5
wget https://paddle-org.bj.bcebos.com/paddlescience/datasets/transformer_physx/cylinder_valid.hdf5
# windows
# curl https://paddle-org.bj.bcebos.com/paddlescience/datasets/transformer_physx/cylinder_training.hdf5 --output cylinder_training.hdf5
# curl https://paddle-org.bj.bcebos.com/paddlescience/datasets/transformer_physx/cylinder_valid.hdf5 --output cylinder_valid.hdf5
python train_enn.py mode=eval EVAL.pretrained_model_path=https://paddle-org.bj.bcebos.com/paddlescience/models/cylinder/cylinder_pretrained.pdparams
python train_transformer.py mode=eval EVAL.pretrained_model_path=https://paddle-org.bj.bcebos.com/paddlescience/models/cylinder/cylinder_transformer_pretrained.pdparams
```
co63oc marked this conversation as resolved.
Show resolved Hide resolved

## 1. 背景简介

圆柱绕流问题可以应用于很多领域。例如，在工业设计中，它可以被用来模拟和优化流体在各种设备中的流动，如风力发电机、汽车和飞机的流体动力学性能等。在环保领域，圆柱绕流问题也有应用，如预测和控制河流的洪水、研究污染物的扩散等。此外，在工程实践中，如流体动力学、流体静力学、热交换、空气动力学等领域，圆柱绕流问题也具有实际意义。
Expand Down Expand Up @@ -113,19 +139,19 @@ $$Re \sim(100, 750)$$

首先展示代码中定义的各个参数变量，每个参数的具体含义会在下面使用到时进行解释。

``` py linenums="50" title="examples/cylinder/2d_unsteady/transformer_physx/train_enn.py"
``` py linenums="58" title="examples/cylinder/2d_unsteady/transformer_physx/train_enn.py"
--8<--
examples/cylinder/2d_unsteady/transformer_physx/train_enn.py:50:65
examples/cylinder/2d_unsteady/transformer_physx/train_enn.py:58:59
--8<--
```

#### 3.2.1 约束构建

本案例基于数据驱动的方法求解问题，因此需要使用 PaddleScience 内置的 `SupervisedConstraint` 构建监督约束。在定义约束之前，需要首先指定监督约束中用于数据加载的各个参数，代码如下：

``` py linenums="70" title="examples/cylinder/2d_unsteady/transformer_physx/train_enn.py"
``` py linenums="61" title="examples/cylinder/2d_unsteady/transformer_physx/train_enn.py"
--8<--
examples/cylinder/2d_unsteady/transformer_physx/train_enn.py:70:87
examples/cylinder/2d_unsteady/transformer_physx/train_enn.py:61:80
--8<--
```

Expand All @@ -144,9 +170,9 @@ examples/cylinder/2d_unsteady/transformer_physx/train_enn.py:70:87

定义监督约束的代码如下：

``` py linenums="89" title="examples/cylinder/2d_unsteady/transformer_physx/train_enn.py"
``` py linenums="82" title="examples/cylinder/2d_unsteady/transformer_physx/train_enn.py"
--8<--
examples/cylinder/2d_unsteady/transformer_physx/train_enn.py:89:97
examples/cylinder/2d_unsteady/transformer_physx/train_enn.py:82:94
--8<--
```

Expand All @@ -169,37 +195,37 @@ examples/cylinder/2d_unsteady/transformer_physx/train_enn.py:89:97

用 PaddleScience 代码表示如下：

``` py linenums="102" title="examples/cylinder/2d_unsteady/transformer_physx/train_enn.py"
``` py linenums="104" title="examples/cylinder/2d_unsteady/transformer_physx/train_enn.py"
--8<--
examples/cylinder/2d_unsteady/transformer_physx/train_enn.py:102:108
examples/cylinder/2d_unsteady/transformer_physx/train_enn.py:104:109
--8<--
```

其中，`CylinderEmbedding` 的前两个参数在前文中已有描述，这里不再赘述，网络模型的第三、四个参数是训练数据集的均值和方差，用于归一化输入数据。计算均值、方差的的代码表示如下：

``` py linenums="29" title="examples/cylinder/2d_unsteady/transformer_physx/train_enn.py"
``` py linenums="32" title="examples/cylinder/2d_unsteady/transformer_physx/train_enn.py"
--8<--
examples/cylinder/2d_unsteady/transformer_physx/train_enn.py:29:46
examples/cylinder/2d_unsteady/transformer_physx/train_enn.py:32:49
--8<--
```

#### 3.2.3 学习率与优化器构建

本案例中使用的学习率方法为 `ExponentialDecay`，学习率大小设置为0.001。优化器使用 `Adam`，梯度裁剪使用了 Paddle 内置的 `ClipGradByGlobalNorm` 方法。用 PaddleScience 代码表示如下：

``` py linenums="110" title="examples/cylinder/2d_unsteady/transformer_physx/train_enn.py"
``` py linenums="111" title="examples/cylinder/2d_unsteady/transformer_physx/train_enn.py"
--8<--
examples/cylinder/2d_unsteady/transformer_physx/train_enn.py:110:124
examples/cylinder/2d_unsteady/transformer_physx/train_enn.py:111:120
--8<--
```

#### 3.2.4 评估器构建

本案例训练过程中会按照一定的训练轮数间隔，使用验证集评估当前模型的训练情况，需要使用 `SupervisedValidator` 构建评估器。代码如下：

``` py linenums="126" title="examples/cylinder/2d_unsteady/transformer_physx/train_enn.py"
``` py linenums="124" title="examples/cylinder/2d_unsteady/transformer_physx/train_enn.py"
--8<--
examples/cylinder/2d_unsteady/transformer_physx/train_enn.py:126:153
examples/cylinder/2d_unsteady/transformer_physx/train_enn.py:124:151
--8<--
```

Expand All @@ -209,39 +235,39 @@ examples/cylinder/2d_unsteady/transformer_physx/train_enn.py:126:153

完成上述设置之后，只需要将上述实例化的对象按顺序传递给 `ppsci.solver.Solver`，然后启动训练、评估。

``` py linenums="156" title="examples/cylinder/2d_unsteady/transformer_physx/train_enn.py"
``` py linenums="153" title="examples/cylinder/2d_unsteady/transformer_physx/train_enn.py"
--8<--
examples/cylinder/2d_unsteady/transformer_physx/train_enn.py:156:
examples/cylinder/2d_unsteady/transformer_physx/train_enn.py:153:169
--8<--
```

### 3.3 Transformer 模型

上文介绍了如何构建 Embedding 模型的训练、评估，在本节中将介绍如何使用训练好的 Embedding 模型训练 Transformer 模型。因为训练 Transformer 模型的步骤与训练 Embedding 模型的步骤基本相似，因此本节在两者的重复部分的各个参数不再详细介绍。首先将代码中定义的各个参数变量展示如下，每个参数的具体含义会在下面使用到时进行解释。

``` py linenums="57" title="examples/cylinder/2d_unsteady/transformer_physx/train_transformer.py"
``` yaml linenums="26" title="examples/cylinder/2d_unsteady/transformer_physx/conf/transformer.yaml"
--8<--
examples/cylinder/2d_unsteady/transformer_physx/train_transformer.py:57:79
examples/cylinder/2d_unsteady/transformer_physx/conf/transformer.yaml:26:33
--8<--
```

#### 3.3.1 约束构建

Transformer 模型同样基于数据驱动的方法求解问题，因此需要使用 PaddleScience 内置的 `SupervisedConstraint` 构建监督约束。在定义约束之前，需要首先指定监督约束中用于数据加载的各个参数，代码如下：

``` py linenums="87" title="examples/cylinder/2d_unsteady/transformer_physx/train_transformer.py"
``` py linenums="68" title="examples/cylinder/2d_unsteady/transformer_physx/train_transformer.py"
--8<--
examples/cylinder/2d_unsteady/transformer_physx/train_transformer.py:87:104
examples/cylinder/2d_unsteady/transformer_physx/train_transformer.py:68:85
--8<--
```

数据加载的各个参数与 Embedding 模型中的基本一致，不再赘述。需要说明的是由于 Transformer 模型训练的输入数据是 Embedding 模型 Encoder 模块的输出数据，因此我们将训练好的 Embedding 模型作为 `CylinderDataset` 的一个参数，在初始化时首先将训练数据映射到编码空间。

定义监督约束的代码如下：

``` py linenums="106" title="examples/cylinder/2d_unsteady/transformer_physx/train_transformer.py"
``` py linenums="87" title="examples/cylinder/2d_unsteady/transformer_physx/train_transformer.py"
--8<--
examples/cylinder/2d_unsteady/transformer_physx/train_transformer.py:106:111
examples/cylinder/2d_unsteady/transformer_physx/train_transformer.py:87:92
--8<--
```

Expand All @@ -256,9 +282,9 @@ examples/cylinder/2d_unsteady/transformer_physx/train_transformer.py:106:111

用 PaddleScience 代码表示如下：

``` py linenums="116" title="examples/cylinder/2d_unsteady/transformer_physx/train_transformer.py"
``` py linenums="98" title="examples/cylinder/2d_unsteady/transformer_physx/train_transformer.py"
--8<--
examples/cylinder/2d_unsteady/transformer_physx/train_transformer.py:116:124
examples/cylinder/2d_unsteady/transformer_physx/train_transformer.py:98:98
--8<--
```

Expand All @@ -268,19 +294,19 @@ examples/cylinder/2d_unsteady/transformer_physx/train_transformer.py:116:124

本案例中使用的学习率方法为 `CosineWarmRestarts`，学习率大小设置为0.001。优化器使用 `Adam`，梯度裁剪使用了 Paddle 内置的 `ClipGradByGlobalNorm` 方法。用 PaddleScience 代码表示如下：

``` py linenums="126" title="examples/cylinder/2d_unsteady/transformer_physx/train_transformer.py"
``` py linenums="100" title="examples/cylinder/2d_unsteady/transformer_physx/train_transformer.py"
--8<--
examples/cylinder/2d_unsteady/transformer_physx/train_transformer.py:126:140
examples/cylinder/2d_unsteady/transformer_physx/train_transformer.py:100:107
--8<--
```

#### 3.3.4 评估器构建

训练过程中会按照一定的训练轮数间隔，使用验证集评估当前模型的训练情况，需要使用 `SupervisedValidator` 构建评估器。用 PaddleScience 代码表示如下：

``` py linenums="142" title="examples/cylinder/2d_unsteady/transformer_physx/train_transformer.py"
``` py linenums="110" title="examples/cylinder/2d_unsteady/transformer_physx/train_transformer.py"
--8<--
examples/cylinder/2d_unsteady/transformer_physx/train_transformer.py:142:168
examples/cylinder/2d_unsteady/transformer_physx/train_transformer.py:110:135
--8<--
```

Expand All @@ -290,25 +316,25 @@ examples/cylinder/2d_unsteady/transformer_physx/train_transformer.py:142:168

在本文中首先定义了对 Transformer 模型输出数据变换到物理状态空间的代码：

``` py linenums="33" title="examples/cylinder/2d_unsteady/transformer_physx/train_transformer.py"
``` py linenums="35" title="examples/cylinder/2d_unsteady/transformer_physx/train_transformer.py"
--8<--
examples/cylinder/2d_unsteady/transformer_physx/train_transformer.py:33:53
examples/cylinder/2d_unsteady/transformer_physx/train_transformer.py:35:56
--8<--
```

``` py linenums="83" title="examples/cylinder/2d_unsteady/transformer_physx/train_transformer.py"
``` py linenums="64" title="examples/cylinder/2d_unsteady/transformer_physx/train_transformer.py"
--8<--
examples/cylinder/2d_unsteady/transformer_physx/train_transformer.py:83:84
examples/cylinder/2d_unsteady/transformer_physx/train_transformer.py:64:65
--8<--
```

可以看到，程序首先载入了训练好的 Embedding 模型，然后在 `OutputTransform` 的 `__call__` 函数内实现了编码向量到物理状态空间的变换。

在定义好了以上代码之后，就可以实现可视化器代码的构建了：

``` py linenums="170" title="examples/cylinder/2d_unsteady/transformer_physx/train_transformer.py"
``` py linenums="146" title="examples/cylinder/2d_unsteady/transformer_physx/train_transformer.py"
--8<--
examples/cylinder/2d_unsteady/transformer_physx/train_transformer.py:170:197
examples/cylinder/2d_unsteady/transformer_physx/train_transformer.py:146:164
--8<--
```

Expand All @@ -318,9 +344,9 @@ examples/cylinder/2d_unsteady/transformer_physx/train_transformer.py:170:197

完成上述设置之后，只需要将上述实例化的对象按顺序传递给 `ppsci.solver.Solver`，然后启动训练、评估。

``` py linenums="199" title="examples/cylinder/2d_unsteady/transformer_physx/train_transformer.py"
``` py linenums="166" title="examples/cylinder/2d_unsteady/transformer_physx/train_transformer.py"
--8<--
examples/cylinder/2d_unsteady/transformer_physx/train_transformer.py:199:
examples/cylinder/2d_unsteady/transformer_physx/train_transformer.py:166:184
--8<--
```

Expand Down
Loading