赛题七-开发grad_fn、next_functions两个API 并暴露到python端-v1 #54838

qiuwenbogdut · 2023-06-25T04:22:37Z

PR types

New features

PR changes

APIs

Description

中国软件开源创新大赛：飞桨框架任务挑战赛-赛题七：实现飞桨框架动态图反向图的可视化

 赛题七 - RFC设计文档
 赛题七- issue tracking

paddle-bot · 2023-06-25T04:22:41Z

你的PR提交成功，感谢你对开源项目的贡献!
请关注后续CI自动化测试结果，详情请参考Paddle-CI手册。
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

wanghuancoder · 2023-06-25T04:46:49Z

paddle/fluid/eager/grad_node_info.cc

@@ -559,4 +559,19 @@ void GradNodeBase::HandleComplexGradToRealGrad(
  }
 }

+std::vector<GradNodeBase*> GradNodeBase::NextFunctions() {


这里返回指针，而不是智能指针是不安全的。如果node析构了，而Python端或者别的地方还持有std::vector<GradNodeBase*>，可能造成非法访问

// 修改方法的返回类型为`std::vector<std::shared_ptr<GradNodeBase>> std::vector<std::shared_ptr<GradNodeBase>> GradNodeBase::NextFunctions() { std::vector<std::shared_ptr<GradNodeBase>> next_nodes; const paddle::small_vector<std::vector<GradSlotMeta>, kSlotSmallVectorSize>& metas = OutputMeta(); for (const auto& meta_list : metas) { for (const GradSlotMeta& meta : meta_list) { const auto& edge = meta.GetEdge(); // 获取关联的智能指针而非裸指针 std::shared_ptr<GradNodeBase> next_node = edge.GetGradNode(); next_nodes.push_back(next_node); } } return next_nodes; }

修改成这样使用智能指针是否可行?

wanghuancoder · 2023-06-25T04:47:08Z

paddle/fluid/eager/grad_node_info.h

+  //增加一个方法 返回一个固定值1 作为测试
+  int64_t GetNextHookId() { return 1; }


这个是测试用的吗？如果是最后得删掉哈~

这个是测试用的吗？如果是最后得删掉哈~

是的, 后面删掉

wanghuancoder · 2023-06-25T04:47:32Z

paddle/fluid/eager/grad_node_info.h

+  //增加一个方法 返回一个固定值1 作为测试
+  int64_t GetNextHookId() { return 1; }
+
+  // 定义一个方法实现访问当前gradnode的下一个gradnode


咱们框架里最好不要有中文注释啊~~改成英文吧~~

咱们框架里最好不要有中文注释啊~~改成英文吧~~

好的后面注释使用英文

wanghuancoder · 2023-06-25T04:47:56Z

paddle/fluid/pybind/eager_properties.cc

+PyObject* tensor_properties_qiutest(TensorObject* self, void* closure) {
+  EAGER_TRY
+  int grad_fn_value = 1;
+  return ToPyObject(grad_fn_value);
+  EAGER_CATCH_AND_THROW_RETURN_NULL
+}


这个应该是测试代码吧？

这个应该是测试代码吧？

是的需要删除

wanghuancoder · 2023-06-25T04:48:31Z

paddle/fluid/pybind/eager_properties.cc

+    // 打印一下meta
+    std::cout << "meta is " << meta << std::endl;


std::cout是不能出现在正式代码里的啊。

std::cout是不能出现在正式代码里的啊。

好的,下次注意

wanghuancoder · 2023-06-25T04:48:52Z

paddle/fluid/pybind/eager_properties.cc

+        auto grad_node = meta->GradNode();        // Convert GradNode to a Python object
+        // The conversion will depend on the structure of GradNode.
+
+        // 实现一下将gradnode 对象转化成python对象


后面所有的中文注释都处理一下吧~

后面所有的中文注释都处理一下吧~

好的

wanghuancoder · 2023-06-25T04:49:34Z

paddle/fluid/pybind/eager_properties.cc

+        //获取python 转换器
+        py::object py_obj = py::cast(grad_node, py::return_value_policy::reference);
+        py::handle py_handle = py::handle(py_obj);
+        PyObject* py_grad_node = py_handle.ptr();
+        Py_INCREF(py_grad_node);  // 增加引用计数，以确保在返回时不会被销毁


把这段逻辑封装一个ToPyobject吧

wanghuancoder · 2023-06-25T04:49:52Z

paddle/fluid/pybind/eager_properties.cc

@@ -320,6 +373,8 @@ struct PyGetSetDef variable_properties[] = {
    {"dtype", (getter)tensor_properties_get_dtype, nullptr, nullptr, nullptr},
    {"type", (getter)tensor_properties_get_type, nullptr, nullptr, nullptr},
    {"is_leaf", (getter)tensor_properties_is_leaf, nullptr, nullptr, nullptr},
+    {"grad_fn", (getter)tensor_properties_get_grad_fn, nullptr, nullptr, nullptr},
+    {"qiutest", (getter)tensor_properties_qiutest, nullptr, nullptr, nullptr},


测试代码删一下

测试代码删一下

ok

wanghuancoder · 2023-06-25T04:50:22Z

paddle/fluid/pybind/pybind.cc

+
+  py::class_<egr::GradNodeBase>(m, "GradNodeBase")
+      //GetNextHookId 暴露一个函数 一直返回一个固定的值
+      .def("get_next_hook_id", &egr::GradNodeBase::GetNextHookId)


这个是干嘛用的？我没太理解~

这个是干嘛用的？我没太理解~
这个也需要删除的

paddle-ci-bot · 2023-07-03T03:10:02Z

Sorry to inform you that 5e2ba5c's CIs have passed for more than 7 days. To prevent PR conflicts, you need to re-run all CIs manually.

CLAassistant · 2023-07-03T10:27:16Z

All committers have signed the CLA.

wanghuancoder · 2023-07-07T02:45:30Z

paddle/fluid/eager/grad_node_info.cc

+      // std::shared_ptr<GradNodeBase>
+      // next_node(edge.GetMutableGradNode().get());


这两行注释删掉吧

wanghuancoder · 2023-07-07T02:47:05Z

paddle/fluid/pybind/eager_properties.cc

+PyObject* convert_grad_node_to_py_object(egr::GradNodeBase* grad_node) {
+  py::object py_obj = py::cast(grad_node, py::return_value_policy::reference);
+  py::handle py_handle = py::handle(py_obj);
+  PyObject* py_grad_node = py_handle.ptr();
+  Py_INCREF(py_grad_node);
+  return py_grad_node;
+}


这个函数重命名成ToPyObject。放到paddle/fluid/pybind/eager_utils.h和paddle/fluid/pybind/eager_utils.cc中吧

好的已经处理

luotao1 · 2023-07-07T03:40:14Z

#54838 (comment)
同时解决下CLA的签署问题

qiuwenbogdut · 2023-07-07T06:28:49Z

#54838 (comment) 同时解决下CLA的签署问题
好的已经解决

wanghuancoder

LGTM

lanxianghit

LGTM for Python API

luotao1 · 2023-07-11T02:27:35Z

hi, @qiuwenbogdut

非常感谢你对飞桨框架的贡献，我们正在运营一个PFCC组织，会通过定期分享技术知识与发布开发者主导任务的形式持续为飞桨框架做贡献，详情可见 https://github.com/luotao1 主页说明。
如果你对PFCC有兴趣，请发送邮件至 [email protected]，我们会邀请你加入~

* [尝试] 给tensor增加一个属性, 这个属性是一个定值 1 * 暴露gradnode 并构建gradnode新的方法(用来测试)进行暴露给python python端可以访问 * 开发grad_fn、next_functions两个API 并暴露到python端- 做一些规范化处理 * 增加一个单元测试 * 优化 code-style

qiuwenbogdut added 2 commits June 6, 2023 18:11

[尝试] 给tensor增加一个属性, 这个属性是一个定值 1

260a4f7

暴露gradnode 并构建gradnode新的方法(用来测试)进行暴露给python python端可以访问

922f79f

paddle-bot bot added contributor External developers status: proposed labels Jun 25, 2023

qiuwenbogdut changed the title ~~开发grad_fn、next_functions两个API 并暴露到python端-v1~~ 赛题七-开发grad_fn、next_functions两个API 并暴露到python端-v1 Jun 25, 2023

qiuwenbogdut mentioned this pull request Jun 25, 2023

赛题七：实现飞桨框架动态图反向图的可视化 Tracking Issue #54347

Closed

9 tasks

wanghuancoder reviewed Jun 25, 2023

View reviewed changes

luotao1 assigned luotao1 and wanghuancoder Jun 26, 2023

SigureMo self-requested a review June 30, 2023 03:31

qiuwenbogdut force-pushed the dev_qiu branch from 1326c6e to 5a6a948 Compare July 4, 2023 02:04

qiuwenbogdut requested a review from wanghuancoder July 4, 2023 04:31

wanghuancoder reviewed Jul 7, 2023

View reviewed changes

开发grad_fn、next_functions两个API 并暴露到python端- 做一些规范化处理

4887161

qiuwenbogdut force-pushed the dev_qiu branch from f23c652 to 4887161 Compare July 7, 2023 06:23

qiuwenbogdut force-pushed the dev_qiu branch from 88c94da to 4887161 Compare July 7, 2023 06:39

qiuwenbogdut added 2 commits July 7, 2023 06:50

增加一个单元测试

f54aff8

Merge branch 'develop' into dev_qiu

09e86f5

wanghuancoder previously approved these changes Jul 7, 2023

View reviewed changes

优化 code-style

fa9384c

qiuwenbogdut dismissed wanghuancoder’s stale review via fa9384c July 7, 2023 11:14

lanxianghit approved these changes Jul 10, 2023

View reviewed changes

wanghuancoder approved these changes Jul 10, 2023

View reviewed changes

dingjiaweiww approved these changes Jul 10, 2023

View reviewed changes

wanghuancoder merged commit ab46b14 into PaddlePaddle:develop Jul 11, 2023

Ligoml mentioned this pull request Jul 24, 2023

中国软件开源创新大赛：飞桨框架任务挑战赛（下） #55663

Closed

qiuwenbogdut mentioned this pull request Dec 5, 2023

WAVE SUMMIT+2023下半年飞桨开源之星评选-信息征集 PaddlePaddle/community#765

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

赛题七-开发grad_fn、next_functions两个API 并暴露到python端-v1 #54838

赛题七-开发grad_fn、next_functions两个API 并暴露到python端-v1 #54838

qiuwenbogdut commented Jun 25, 2023

paddle-bot bot commented Jun 25, 2023

wanghuancoder Jun 25, 2023

qiuwenbogdut Jun 25, 2023

wanghuancoder Jun 25, 2023

qiuwenbogdut Jun 25, 2023

wanghuancoder Jun 25, 2023

qiuwenbogdut Jun 25, 2023

wanghuancoder Jun 25, 2023

qiuwenbogdut Jun 25, 2023

wanghuancoder Jun 25, 2023

qiuwenbogdut Jun 25, 2023

wanghuancoder Jun 25, 2023

qiuwenbogdut Jun 25, 2023

wanghuancoder Jun 25, 2023

wanghuancoder Jun 25, 2023

qiuwenbogdut Jun 25, 2023

wanghuancoder Jun 25, 2023

qiuwenbogdut Jun 25, 2023

paddle-ci-bot bot commented Jul 3, 2023

CLAassistant commented Jul 3, 2023 •

edited

Loading

wanghuancoder Jul 7, 2023

wanghuancoder Jul 7, 2023

qiuwenbogdut Jul 7, 2023

luotao1 commented Jul 7, 2023

qiuwenbogdut commented Jul 7, 2023

wanghuancoder left a comment

lanxianghit left a comment

luotao1 commented Jul 11, 2023

		//增加一个方法返回一个固定值1 作为测试
		int64_t GetNextHookId() { return 1; }

		// 打印一下meta
		std::cout << "meta is " << meta << std::endl;

		// std::shared_ptr<GradNodeBase>
		// next_node(edge.GetMutableGradNode().get());

赛题七-开发grad_fn、next_functions两个API 并暴露到python端-v1 #54838

赛题七-开发grad_fn、next_functions两个API 并暴露到python端-v1 #54838

Conversation

qiuwenbogdut commented Jun 25, 2023

PR types

PR changes

Description

paddle-bot bot commented Jun 25, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

paddle-ci-bot bot commented Jul 3, 2023

CLAassistant commented Jul 3, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

luotao1 commented Jul 7, 2023

qiuwenbogdut commented Jul 7, 2023

wanghuancoder left a comment

Choose a reason for hiding this comment

lanxianghit left a comment

Choose a reason for hiding this comment

luotao1 commented Jul 11, 2023

CLAassistant commented Jul 3, 2023 •

edited

Loading