Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Include validation statement #446

Merged
merged 4 commits into from
Oct 15, 2024
Merged

Include validation statement #446

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
9a717d6
added validation blurb
gowerc Oct 11, 2024
6296da9
Update vignettes/FAQ.Rmd
gowerc Oct 15, 2024
baa8168
Update vignettes/FAQ.Rmd
gowerc Oct 15, 2024
e89ed45
updated html vignette
gowerc Oct 15, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,11 @@ The basic usage of these core functions is described in the quickstart vignette:
vignette(topic = "quickstart", package = "rbmi")
```

## Validation

For clarification on the current validation status of rbmi please see the FAQ vignette.


## Support

For any help with regards to using the package or if you find a bug please create a [GitHub issue](https://github.com/insightsengineering/rbmi/issues)
Expand Down
22 changes: 22 additions & 0 deletions vignettes/FAQ.Rmd
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,28 @@ knitr::opts_chunk$set(
This document provides answers to common questions about the `rbmi` package.
It is intended to be read after the `rbmi: Quickstart` vignette.

<br>


## Is `rbmi` validated ?
gowerc marked this conversation as resolved.
Show resolved Hide resolved

With regards to software, validation is the act of ensuring that the software meets the needs and
gowerc marked this conversation as resolved.
Show resolved Hide resolved
requirements of users given the conditions of actual use. The FDA provides general principles and
guidance for validation but leaves it to individual sponsors to define their specific validation
processes. Therefore, no individual R package can claim to be 'validated' independently, as
validation depends on the entire software stack and the specific processes of each company.

That being said, some of the core components of any validation process are the design specification
(what is the software supposed to do) as well as the testing / test results that demonstrate that
the design specification has been met. For rbmi, the design specification is documented extensively,
gowerc marked this conversation as resolved.
Show resolved Hide resolved
both at a macro level in vignettes and literature publications, and at a micro level in detailed
function manuals. This is supported by our extensive suite of unit and integration tests, which
ensure the software consistently produces correct output across a wide range of input scenarios.

This documentation and test coverage enable rbmi to be easily installed and integrated into any
gowerc marked this conversation as resolved.
Show resolved Hide resolved
R system, in alignment with the system's broader validation process.


<br>

## How do the methods in `rbmi` compare to the mixed model for repeated measures (MMRM) implemented in the `mmrm` package?
Expand Down
48 changes: 33 additions & 15 deletions vignettes/FAQ.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -355,11 +355,12 @@ <h4 class="author">Alessandro Noci, Craig Gower-Page and Marcel Wolbers</h4>
<ul>
<li><a href="#introduction" id="toc-introduction"><span class="toc-section-number">1</span> Introduction</a>
<ul>
<li><a href="#how-do-the-methods-in-rbmi-compare-to-the-mixed-model-for-repeated-measures-mmrm-implemented-in-the-mmrm-package" id="toc-how-do-the-methods-in-rbmi-compare-to-the-mixed-model-for-repeated-measures-mmrm-implemented-in-the-mmrm-package"><span class="toc-section-number">1.1</span> How do the methods in <code>rbmi</code> compare to the mixed model for repeated measures (MMRM) implemented in the <code>mmrm</code> package?</a></li>
<li><a href="#how-does-rbmi-compare-to-general-purpose-software-for-multiple-imputation-mi-such-as-mice" id="toc-how-does-rbmi-compare-to-general-purpose-software-for-multiple-imputation-mi-such-as-mice"><span class="toc-section-number">1.2</span> How does <code>rbmi</code> compare to general-purpose software for multiple imputation (MI) such as <code>mice</code>?</a></li>
<li><a href="#how-to-handle-missing-data-in-baseline-covariates-in-rbmi" id="toc-how-to-handle-missing-data-in-baseline-covariates-in-rbmi"><span class="toc-section-number">1.3</span> How to handle missing data in baseline covariates in <code>rbmi</code>?</a></li>
<li><a href="#why-does-rbmi-by-default-use-an-ancova-analysis-model-and-not-an-mmrm-analysis-model" id="toc-why-does-rbmi-by-default-use-an-ancova-analysis-model-and-not-an-mmrm-analysis-model"><span class="toc-section-number">1.4</span> Why does <code>rbmi</code> by default use an ANCOVA analysis model and not an MMRM analysis model?</a></li>
<li><a href="#how-can-i-analyse-the-change-from-baseline-in-the-analysis-model-when-imputation-was-done-on-the-original-outcomes" id="toc-how-can-i-analyse-the-change-from-baseline-in-the-analysis-model-when-imputation-was-done-on-the-original-outcomes"><span class="toc-section-number">1.5</span> How can I analyse the change-from-baseline in the analysis model when imputation was done on the original outcomes?</a></li>
<li><a href="#is-rbmi-validated" id="toc-is-rbmi-validated"><span class="toc-section-number">1.1</span> Is <code>rbmi</code> validated ?</a></li>
<li><a href="#how-do-the-methods-in-rbmi-compare-to-the-mixed-model-for-repeated-measures-mmrm-implemented-in-the-mmrm-package" id="toc-how-do-the-methods-in-rbmi-compare-to-the-mixed-model-for-repeated-measures-mmrm-implemented-in-the-mmrm-package"><span class="toc-section-number">1.2</span> How do the methods in <code>rbmi</code> compare to the mixed model for repeated measures (MMRM) implemented in the <code>mmrm</code> package?</a></li>
<li><a href="#how-does-rbmi-compare-to-general-purpose-software-for-multiple-imputation-mi-such-as-mice" id="toc-how-does-rbmi-compare-to-general-purpose-software-for-multiple-imputation-mi-such-as-mice"><span class="toc-section-number">1.3</span> How does <code>rbmi</code> compare to general-purpose software for multiple imputation (MI) such as <code>mice</code>?</a></li>
<li><a href="#how-to-handle-missing-data-in-baseline-covariates-in-rbmi" id="toc-how-to-handle-missing-data-in-baseline-covariates-in-rbmi"><span class="toc-section-number">1.4</span> How to handle missing data in baseline covariates in <code>rbmi</code>?</a></li>
<li><a href="#why-does-rbmi-by-default-use-an-ancova-analysis-model-and-not-an-mmrm-analysis-model" id="toc-why-does-rbmi-by-default-use-an-ancova-analysis-model-and-not-an-mmrm-analysis-model"><span class="toc-section-number">1.5</span> Why does <code>rbmi</code> by default use an ANCOVA analysis model and not an MMRM analysis model?</a></li>
<li><a href="#how-can-i-analyse-the-change-from-baseline-in-the-analysis-model-when-imputation-was-done-on-the-original-outcomes" id="toc-how-can-i-analyse-the-change-from-baseline-in-the-analysis-model-when-imputation-was-done-on-the-original-outcomes"><span class="toc-section-number">1.6</span> How can I analyse the change-from-baseline in the analysis model when imputation was done on the original outcomes?</a></li>
</ul></li>
</ul>
</div>
Expand All @@ -369,8 +370,25 @@ <h1><span class="header-section-number">1</span> Introduction</h1>
<p>This document provides answers to common questions about the <code>rbmi</code> package.
It is intended to be read after the <code>rbmi: Quickstart</code> vignette.</p>
<p><br></p>
<div id="how-do-the-methods-in-rbmi-compare-to-the-mixed-model-for-repeated-measures-mmrm-implemented-in-the-mmrm-package" class="section level2" number="1.1">
<h2><span class="header-section-number">1.1</span> How do the methods in <code>rbmi</code> compare to the mixed model for repeated measures (MMRM) implemented in the <code>mmrm</code> package?</h2>
<div id="is-rbmi-validated" class="section level2" number="1.1">
<h2><span class="header-section-number">1.1</span> Is <code>rbmi</code> validated ?</h2>
<p>With regards to software, validation is the act of ensuring that the software meets the needs and
requirements of users given the conditions of actual use. The FDA provides general principles and
guidance for validation but leaves it to individual sponsors to define their specific validation
processes. Therefore, no individual R package can claim to be ‘validated’ independently, as
validation depends on the entire software stack and the specific processes of each company.</p>
<p>That being said, some of the core components of any validation process are the design specification
(what is the software supposed to do) as well as the testing / test results that demonstrate that
the design specification has been met. For rbmi, the design specification is documented extensively,
both at a macro level in vignettes and literature publications, and at a micro level in detailed
function manuals. This is supported by our extensive suite of unit and integration tests, which
ensure the software consistently produces correct output across a wide range of input scenarios.</p>
<p>This documentation and test coverage enable rbmi to be easily installed and integrated into any
R system, in alignment with the system’s broader validation process.</p>
<p><br></p>
</div>
<div id="how-do-the-methods-in-rbmi-compare-to-the-mixed-model-for-repeated-measures-mmrm-implemented-in-the-mmrm-package" class="section level2" number="1.2">
<h2><span class="header-section-number">1.2</span> How do the methods in <code>rbmi</code> compare to the mixed model for repeated measures (MMRM) implemented in the <code>mmrm</code> package?</h2>
<p><code>rbmi</code> was designed to complement and, occasionally, replace standard MMRM analyses for clinical trials with longitudinal endpoints.</p>
<p><strong>Strengths</strong> of <code>rbmi</code> compared to the standard MMRM model are:</p>
<ul>
Expand All @@ -384,8 +402,8 @@ <h2><span class="header-section-number">1.1</span> How do the methods in <code>r
</ul>
<p><br></p>
</div>
<div id="how-does-rbmi-compare-to-general-purpose-software-for-multiple-imputation-mi-such-as-mice" class="section level2" number="1.2">
<h2><span class="header-section-number">1.2</span> How does <code>rbmi</code> compare to general-purpose software for multiple imputation (MI) such as <code>mice</code>?</h2>
<div id="how-does-rbmi-compare-to-general-purpose-software-for-multiple-imputation-mi-such-as-mice" class="section level2" number="1.3">
<h2><span class="header-section-number">1.3</span> How does <code>rbmi</code> compare to general-purpose software for multiple imputation (MI) such as <code>mice</code>?</h2>
<p><code>rbmi</code> covers only “MMRM-type” settings, i.e. settings with a single longitudinal continuous outcome which may be missing at some visits and hence require imputation.</p>
<p>For these settings, it has several <strong>advantages</strong> over general-purpose MI software:</p>
<ul>
Expand All @@ -396,14 +414,14 @@ <h2><span class="header-section-number">1.2</span> How does <code>rbmi</code> co
<p>However, <code>rbmi</code> is much more limited in its functionality than general-purpose MI software.</p>
<p><br></p>
</div>
<div id="how-to-handle-missing-data-in-baseline-covariates-in-rbmi" class="section level2" number="1.3">
<h2><span class="header-section-number">1.3</span> How to handle missing data in baseline covariates in <code>rbmi</code>?</h2>
<div id="how-to-handle-missing-data-in-baseline-covariates-in-rbmi" class="section level2" number="1.4">
<h2><span class="header-section-number">1.4</span> How to handle missing data in baseline covariates in <code>rbmi</code>?</h2>
<p><code>rbmi</code> does not support imputation of missing baseline covariates. Therefore, missing baseline covariates need to be handled outside of <code>rbmi</code>.
The best approach for handling missing baseline covariates needs to be made on a case-by-case basis but in the context of randomized trials, relatively simple approach are often sufficient (<span class="citation">White and Thompson (<a href="#ref-White2005">2005</a>)</span>).</p>
<p><br></p>
</div>
<div id="why-does-rbmi-by-default-use-an-ancova-analysis-model-and-not-an-mmrm-analysis-model" class="section level2" number="1.4">
<h2><span class="header-section-number">1.4</span> Why does <code>rbmi</code> by default use an ANCOVA analysis model and not an MMRM analysis model?</h2>
<div id="why-does-rbmi-by-default-use-an-ancova-analysis-model-and-not-an-mmrm-analysis-model" class="section level2" number="1.5">
<h2><span class="header-section-number">1.5</span> Why does <code>rbmi</code> by default use an ANCOVA analysis model and not an MMRM analysis model?</h2>
<p>The theoretical justification for the conditional mean imputation method requires that the analysis model leads to a point estimator which is a linear function of the outcome vector (<span class="citation">Wolbers et al. (<a href="#ref-Wolbers2021">2022</a>)</span>). This is the case for ANCOVA but not for general MMRM models. For the other imputation methods, both ANCOVA and MMRM are valid analysis methods. An MMRM analysis model could be implemented by providing a custom analysis function to the <code>analyse()</code> function.</p>
<p>For further expalanations, we also cite the end of section 2.4 of the conditional mean imputation paper (<span class="citation">Wolbers et al. (<a href="#ref-Wolbers2021">2022</a>)</span>):</p>
<blockquote>
Expand All @@ -414,8 +432,8 @@ <h2><span class="header-section-number">1.4</span> Why does <code>rbmi</code> by
</blockquote>
<p><br></p>
</div>
<div id="how-can-i-analyse-the-change-from-baseline-in-the-analysis-model-when-imputation-was-done-on-the-original-outcomes" class="section level2" number="1.5">
<h2><span class="header-section-number">1.5</span> How can I analyse the change-from-baseline in the analysis model when imputation was done on the original outcomes?</h2>
<div id="how-can-i-analyse-the-change-from-baseline-in-the-analysis-model-when-imputation-was-done-on-the-original-outcomes" class="section level2" number="1.6">
<h2><span class="header-section-number">1.6</span> How can I analyse the change-from-baseline in the analysis model when imputation was done on the original outcomes?</h2>
<p>This can be achieved using custom analysis functions as outlined in Section 7 of the Advanced Vignette. e.g.</p>
<div class="sourceCode" id="cb1"><pre class="sourceCode r"><code class="sourceCode r"><span id="cb1-1"><a href="#cb1-1" tabindex="-1"></a>ancova_modified <span class="ot">&lt;-</span> <span class="cf">function</span>(data, ...) {</span>
<span id="cb1-2"><a href="#cb1-2" tabindex="-1"></a> data2 <span class="ot">&lt;-</span> data <span class="sc">%&gt;%</span> <span class="fu">mutate</span>(<span class="at">ENDPOINT =</span> ENDPOINT <span class="sc">-</span> BASELINE)</span>
Expand Down
Loading