Unable to get protobuf to/from JSON to work using benthos helm chart

[yseleena]

Hi, I am trying to add a processor to convert protobuf messages to JSON. I'm very new to all of this so apologies if I'm missing something obvious. Im trying to follow the protobuf section in the docs

Here is my test proto file (using example from benthos docs):

syntax = "proto3";
package testing;

import "google/protobuf/timestamp.proto";

message Person {
  string first_name = 1;
  string last_name = 2;
  string full_name = 3;
  int32 age = 4;
  int32 id = 5; // Unique ID number for this person.
  string email = 6;

  google.protobuf.Timestamp last_updated = 7;
}

Config Map (I've kept the default and added a .Files.Get to pull the .proto file in):

apiVersion: v1
kind: ConfigMap
metadata:
  name: {{ template "benthos.fullname" . }}-config
  namespace: {{ .Release.Namespace }}
  labels:
    {{- include "benthos.labels" . | nindent 4 }}
data:
  proto.conf: {{ .Files.Get "proto/test.proto" | indent 4 }}
  benthos.yaml: | ...

Within the values.yaml file, I've mounted the .proto file:

extraVolumes: 
  - name: proto-config
    configMap:
      name: benthos-config
      items:
        - key: proto.conf
          path: test.proto

extraVolumeMounts: 
  - name: proto-config
    mountPath: /mnt
    readOnly: true

And then here is my config in the values file:

config: |
  input:
    label: "kafka"
    kafka:
      addresses: 
      - my-cluster-kafka-bootstrap:9092
      topics: 
      - benthos
      consumer_group: "benthos-data"

  pipeline:
    processors:
      - label: protobuf
        protobuf:
          operator: from_json
          message: testing.Person
          import_paths: [ mnt ]

  output:
    broker:
      pattern: fan_out
      outputs: 
      - label: "stdout"
        stdout:
          codec: lines

Those are the only changes I've made, everything else in the helm chart is default values. When I try to run it, I get the following error in the Benthos pod logs:
"Service closing due to: failed to init processor 'protobuf' path root.pipeline.processors.0: failed to parse .proto file: test.proto:3:1: duplicate symbol testing.Person: already defined as message in \"..2023_09_19_16_29_01.3183615336/test.proto\""

I can see that the file is mounted and have tried to follow the docs but not sure what the issue is here. Any help or guidance would be appreciated!

[mihaitodor]

Hey @yseleena 👋 There's a mix of stuff in there and I'm not sure which part of your setup is broken. Since your error mentions "duplicate symbol testing.Person", I'd expect that to be due to somehow including the same message Person declaration twice, either in your test.proto or maybe in another .proto file that you're somehow including. Are you able to reproduce this issue outside of Kubernetes so I can try to experiment with it myself?