Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

kernel panic RGA_BLIT with yolov5 #133

Open
MattiaGrigoli opened this issue Dec 13, 2024 · 0 comments
Open

kernel panic RGA_BLIT with yolov5 #133

MattiaGrigoli opened this issue Dec 13, 2024 · 0 comments

Comments

@MattiaGrigoli
Copy link

MattiaGrigoli commented Dec 13, 2024

i'm trying to build an application that takes a video feed from a camera (with gstreamer) and makes and inference (object detection) with yolov5 using the NPU, i need 4 istances of this application (for 4 camera in total)
the part of gstreamer works flawlessly because with 4 terminal running the gstreamer pipeline (with hardware acceleration) it doesn't run into a kernel panic.
the problem is when i also run the inference:

  • with one inference (one camera) it runs without any problem
  • with two inferences (or more) it goes straight to a kernel panic

the only solution i found on it is to limit the RAM of the board to 4GB, but for me is too little for what i have to do (in the future there will be other things beside the inference)
i'm using rknpu2 to use the NPU.
i know that the issue is generated by the RGA (this explains the weird reason of the 4GB of memory) but i don't know how to fix it. Maybe some kind of dma32 allocation but i have no idea for how to implement it.
is there a fix for my issue?
here are some info:

  • kernel panic
    image

  • error of program
    image

  • code that handles the inference:

`
#pragma once
#include
#include

#include "yolov5.h"
#include "image_utils.h"
#include "file_utils.h"
#include "image_drawing.h"

#include "dma_alloc.hpp"

class YoloModel {

public:
bool loaded = false;
rknn_app_context_t rknn_app_ctx;

YoloModel(std::string model_path) {
int ret;
memset(&rknn_app_ctx, 0, sizeof(rknn_app_context_t));

init_post_process();
ret = init_yolov5_model(model_path.c_str(), &rknn_app_ctx);
if (ret != 0)
{
    printf("init_yolov5_model fail! ret=%d model_path=%s\n", ret, model_path);
    return;
}
loaded = true;

}

bool infer(cv::Mat cvImage) {
if(!loaded)
return false;

  int ret;

  image_buffer_t src_image;
  memset(&src_image, 0, sizeof(image_buffer_t));
  //ret = read_image("../yolov5/model/bus.jpg", &src_image);
  

  /*ret = dma_buf_alloc(DMA_HEAP_DMA32_PATCH, src_image.size, &rknn_app_ctx.img_dma_buf.dma_buf_fd, 
                   (void **) & (rknn_app_ctx.img_dma_buf.dma_buf_virt_addr));
  memcpy(rknn_app_ctx.img_dma_buf.dma_buf_virt_addr, src_image.virt_addr, src_image.size);
  dma_sync_cpu_to_device(rknn_app_ctx.img_dma_buf.dma_buf_fd);
  free(src_image.virt_addr);
  src_image.virt_addr = (unsigned char *)rknn_app_ctx.img_dma_buf.dma_buf_virt_addr;
  src_image.fd = rknn_app_ctx.img_dma_buf.dma_buf_fd;
  rknn_app_ctx.img_dma_buf.size = src_image.size;*/

  src_image.width = cvImage.cols;
  src_image.height = cvImage.rows;
  src_image.format = IMAGE_FORMAT_RGB888;
  src_image.virt_addr = cvImage.data;

  object_detect_result_list od_results;
  ret = inference_yolov5_model(&rknn_app_ctx, &src_image, &od_results);
  if (ret != 0)
  {
      printf("init_yolov5_model fail! ret=%d\n", ret);
      return false;
  }

  // print
  char text[256];
  for (int i = 0; i < od_results.count; i++)
  {
      object_detect_result *det_result = &(od_results.results[i]);
      printf("\t%s @ (%d %d %d %d) %.3f\n", coco_cls_to_name(det_result->cls_id),
            det_result->box.left, det_result->box.top,
            det_result->box.right, det_result->box.bottom,
            det_result->prop);
      int x1 = det_result->box.left;
      int y1 = det_result->box.top;
      int x2 = det_result->box.right;
      int y2 = det_result->box.bottom;

      draw_rectangle(&src_image, x1, y1, x2 - x1, y2 - y1, COLOR_BLUE, 3);
      // sprintf(text, "%s %.1f%%", coco_cls_to_name(det_result->cls_id), det_result->prop * 100);
      // draw_text(&src_image, text, x1, y1 - 20, COLOR_RED, 10);
  }
  /*dma_buf_free(rknn_app_ctx.img_dma_buf.size, &rknn_app_ctx.img_dma_buf.dma_buf_fd, 
            rknn_app_ctx.img_dma_buf.dma_buf_virt_addr);*/
  return true;

}

};
`

  • board = orange pi 5 plus
  • version = ubuntu jammy (joshua)
  • kernel = 5.10 (joshua)

as you can see i've tried to use the dma allocation from the example but it gave me this error:
image
i'm also available to modify the source code of the kernel if needed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant