You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I have many experiments in mind where I need to condition a Transformer Decoder with some input (e.g. image features, discrete binary labels, a one-hot representing some concept, a question, etc.) in order to generate an output (e.g. a report, an answer). I have already implemented many of these ideas using my own custom Transformer Decoder based on PyTorch's standard implementation. However, now I would like to leverage existing pre-trained language models, instead of my custom implementation that always starts from scratch. Thus, I was wondering if there is an easy way to adapt CXR-BERT (or any other model that you guys would recommend) for text generation, given some input. For example, let's say I have a binary vector encoding certain information, and I want to fine-tune CXR-BERT to generate a paragraph verbalizing the information contained in this binary vector. The paragraph could be, for example, a radiology report, so it makes sense that fine-tuning a model like CXR-BERT for report generation should outperform a custom Transformer Decoder from PyTorch trained from scratch.
Questions:
Is this something that can be easily accomplished?
Are there examples of adapting CXR-BERT for text generation?
What if I need a custom input that conditions the text generation, such as a binary vector?
Thank you very much in advance.
The text was updated successfully, but these errors were encountered:
I have many experiments in mind where I need to condition a Transformer Decoder with some input (e.g. image features, discrete binary labels, a one-hot representing some concept, a question, etc.) in order to generate an output (e.g. a report, an answer). I have already implemented many of these ideas using my own custom Transformer Decoder based on PyTorch's standard implementation. However, now I would like to leverage existing pre-trained language models, instead of my custom implementation that always starts from scratch. Thus, I was wondering if there is an easy way to adapt CXR-BERT (or any other model that you guys would recommend) for text generation, given some input. For example, let's say I have a binary vector encoding certain information, and I want to fine-tune CXR-BERT to generate a paragraph verbalizing the information contained in this binary vector. The paragraph could be, for example, a radiology report, so it makes sense that fine-tuning a model like CXR-BERT for report generation should outperform a custom Transformer Decoder from PyTorch trained from scratch.
Questions:
Thank you very much in advance.
The text was updated successfully, but these errors were encountered: