Power and Design

---

section {
    height: 600px;
    width: 60%;
    margin: auto;
    border-radius: 21px;
    background-color: #212121;
}

.remark-slide-container {
background: #212121;
}

.hljs-github .hljs {
    background: transparent;
    color: #b2dfdb;
}

.hljs-github .hljs-keyword {
    color: #64b5f6;
}

.hljs-github .hljs-literal {
    color: #64b5f6;
}

.hljs-github .hljs-number {
    color: #64b5f6;
}

.hljs-github .hljs-string {
    color: #b7b3ef;
}

.hljs-github .hljs {
    background: transparent;
    color: #b2dfdb;
}

.hljs-github .hljs-keyword {
    color: #64b5f6;
}

.hljs-github .hljs-literal {
    color: #64b5f6;
}

.hljs-github .hljs-number {
    color: #64b5f6;
}

.hljs-github .hljs-string {
    color: #b7b3ef;
}

section p {
    text-align: center;
    font-size: 30px;
    background-color: #212121;
    border-radius: 21px;
    font-family: Roboto Condensed;
    font-style: bold;
    padding: 12px;
    color: #bff4ee;
    margin: auto;
}

#center {
text-align: center;
}

#right {
  text-align: right;
}

.center p {
  margin: 0;
  position: absolute;
  top: 50%;
  left: 50%;
  -ms-transform: translate(-50%, -50%);
  transform: translate(-50%, -50%);
}

.center2 {
  margin: 0;
  position: absolute;
  top: 50%;
  left: 50%;
  -ms-transform: translate(-50%, -50%);
  transform: translate(-50%, -50%);
}

.tab {
    display: inline-block;
    margin-left: 40px;
}

.tabdbl {
    display: inline-block;
    margin-left: 80px;
}

.tabtpl {
    display: inline-block;
    margin-left: 120px;
}

.obr
{
    display:block;
    margin-top:-15px;
}

.pull-left-left {
  float: left;
  width: 27%;
}

.pull-right-right {
  float: right;
  width: 32%;
}

img.expand:hover {
  margin: 0 auto;
  position: relative;
  width: 50%;
  display: flex;
  justify-content: center;
  align-items: center;
  align-content: center;
  transform: scale(1.5)             
  translateX(-35%);
  z-index: 99;
  transition:all 0.5s ease-in-out;
  -webkit-transition:all 0.2s ease-in-out;
}

.vertline {
  border-left: 5px solid #212121;
  height: 100px;
  margin-left: 15px;
  margin-right: 15px;
}

*, *:before, *:after {
	 box-sizing: border-box;
	 outline: none;
}

.hover {
	 position: relative;
	 display: flex;
	 align-items: center;
	 justify-content: center;
	 width: 400px;
	 height: 65px;
	 background-color: #e3c0ff;
	 border-radius: 99px;
	 box-shadow: 0 1px 3px rgba(0, 0, 0, 0.12), 0 1px 2px rgba(0, 0, 0, 0.24);
	 transition: all 0.3s cubic-bezier(0.25, 0.8, 0.25, 1);
	 overflow: hidden;
}

.hover:before, .hover:after {
	 position: absolute;
	 top: 0;
	 display: flex;
	 align-items: center;
   justify-content: center;
	 width: 50%;
	 height: 100%;
	 transition: 0.25s linear;
	 z-index: 1;
}

.hover:before {
	 content: '';
	 left: 0;
	 background-color: #ca86ec;
   color: #212121;
}

.hover:after {
	 content: '';
	 right: 0;
	 background-color: #d896ff;
}

.hover:hover {
	 background-color: #cc8bff;
	 box-shadow: 0 14px 28px rgba(0, 0, 0, 0.25), 0 10px 10px rgba(0, 0, 0, 0.22);
}

.hover:hover span {
	 opacity: 0;
	 z-index: -3;
}

.hover:hover:before {
	 opacity: 0.5;
	 transform: translateY(-100%);
}

.hover:hover:after {
	 opacity: 0.5;
	 transform: translateY(100%);
}

.hover span {
	 position: absolute;
	 top: 0;
	 left: 0;
	 display: flex;
	 align-items: center;
	 justify-content: center;
	 text-align: center;
	 width: 100%;
	 height: 100%;
	 color: #212121;
	 font-size: 24px;
	 font-weight: 700;
	 opacity: 1;
	 transition: opacity 0.25s;
	 z-index: 2;
   white-space:pre;
}

.hover .doc-link {
	 position: relative;
	 display: flex;
	 align-items: center;
	 justify-content: center;
	 text-align: center;
	 width: 25%;
	 height: 100%;
	 color: whitesmoke;
	 font-size: 24px;
	 text-decoration: none;
	 transition: 0.25s;
}
 .hover .doc-link i {
	 text-shadow: 1px 1px rgba(70, 98, 127, 0.7);
	 transform: scale(1);
}
 .hover .doc-link:hover {
	 background-color: rgba(245, 245, 245, 0.1);
}
 .hover .doc-link:hover i {
	 animation: bounce 0.4s linear;
}
 @keyframes bounce {
	 40% {
		 transform: scale(1.4);
	}
	 60% {
		 transform: scale(0.8);
	}
	 80% {
		 transform: scale(1.2);
	}
	 100% {
		 transform: scale(1);
	}
}

.boxl {
    width: 50%;
    margin: 5px;
    text-align: center;
}

.boxr {
    margin: 5px;
    text-align: center;
}

.picr {
    display: flex;
    justify-content: space-around;
    align-items: center;
}
</style>

<div>
<style type="text/css">.xaringan-extra-logo {
width: 110px;
height: 128px;
z-index: 0;
background-image: url(/Users/skynet/Documents/WVU/Teaching/GitHub.nosync/edp619/static/img/course_hex.png);
background-size: contain;
background-repeat: no-repeat;
position: absolute;
top:1em;right:1em;
}
</style>
<script>(function () {
  let tries = 0
  function addLogo () {
    if (typeof slideshow === 'undefined') {
      tries += 1
      if (tries < 10) {
        setTimeout(addLogo, 100)
      }
    } else {
      document.querySelectorAll('.remark-slide-content:not(.title-slide):not(.inverse):not(.hide_logo)')
        .forEach(function (slide) {
          const logo = document.createElement('a')
          logo.classList = 'xaringan-extra-logo'
          logo.href = 'https://edp619.asocialdatascientist.com'
          slide.appendChild(logo)
        })
    }
  }
  document.addEventListener('DOMContentLoaded', addLogo)
})()</script>
</div>

---

---

# Welcome!

This is an absolutely minimalist overview of power analysis and the reasons it matters. If you want a deeper dive, consider enrolling in an advanced statistics course like [Power and Sample Size for Multilevel and Longitudinal Study Designs](https://www.coursera.org/learn/power-sample-size) for free. In the meantime, remember that you can explore power and its relationships interactively at [Understanding Statistical Power and Significance Testing](https://rpsychologist.com/d3/nhst/).

Additionally you may notice the following icons in the footnotes. These contain links to external sites that provide extra materials that may be of interest to you. 
<br>
<br>
<br>
<br>
<center>
<div class='footsbs'>
<img src="img/htmlcon-ico.png" alt="HTML icon" width='70' style="padding-right: 20px">
<img src="img/pdfcon-ico.png" alt="PDF icon" width='70' style="padding-right: 20px">
<img src="img/rmdcon-ico.png" alt="Rmarkdown icon" width='70' style="padding-right: 20px">
<img src="img/Rscriptcon-ico.png" alt="Rscript icon" width='70' style="padding-right: 20px">
<img src="img/videocon-ico.png" alt="Video icon" width='70'>
</div>
</center>

---

# Something Important

.center2[<span style="color:#8ec9bb; font-weight: bold; font-style: italic;">(Statistical) Power</span> and its corresponding analysis are by far one of the most misunderstood concepts, in that a lot of people think they know what it is but simply miss the point.]

.footnote[Please note that you will need to recall some concepts covered in an introductory statistics course]

---

# Prerequisites

Before going ahead, make sure that you have a basic understanding of sampling and hypothesis testing. For a refresher, please take a look at both reviews on the next page

---

## Review

If you would like a deeper dive on either area, take a look at the videos below
<br>
<br>
<br>

***Sampling***
</center>
<p align="center">
    <iframe width="450" height="252" src="https://www.youtube.com/embed/67zCIqdeXpo" title="Sample Size and Effective Sample Size, Clearly Explained!!!" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>
</p>
]

***Hypothesis Testing***
</center>
<p align="center">
    <iframe width="450" height="252" src="https://www.youtube.com/embed/0oc49DyA3hU" title="Hypothesis Testing and The Null Hypothesis, Clearly Explained!!!" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>
</p>
]

---

# <span style="color:#8ec9bb; font-weight: bold; font-style: italic;">(Statistical) Power</span>

---

# Definition

.center2[<span style="color:#8ec9bb; font-weight: bold; font-style: italic;">(Statistical) Power</span> is the probability of avoiding a Type II error <i>aka</i> detecting an effect if it exists]

---

# Dodging False Negatives

---

# Influences

> Effect size

> Sample size

> Significance level

---

## Some Things You Should Be Aware of

.pull-right[
This means if there are actual effects detected in 100 different studies with 80% power, then 80 out of 100 statistical tests will actually detect them
]

.pull-right[
If you have an *overpowered* design, you may detect very small effects that are of no practical relevance<sup>1</sup>
]

.footnote[The real problem isn't that *overpowered* experiments may reveal tiny significant effects, rather it is that many academic fields and popular science reporting standard in general emphasize "statistical significance" - *a meaningless term itself* - over effect sizes - aka practical significance. And if you're wondering, by "statistical significance" I mean the *p*-value which is [generally garbage](https://www.tandfonline.com/doi/full/10.1080/00031305.2019.1583913), should be used sparingly, and if you must use it, never report the findings by themselves because they are useless]

.pull-left[
We want the Type I error rate to be reasonably low which is why you often see a default of `$\alpha = 0.05$`
]

.pull-right[
We want power to be high enough to minimize a Type II error ( `$\beta$` ) so we reduce the chance of missing an effect
]

---

# Using <span style="color:#8ec9bb; font-weight: bold; font-style: italic;">(Statistical) Power</span>

.center2[A ***power analysis*** is the procedure that researchers can use to determine if the test contains enough power to make a reasonable conclusion. It can also be used to calculate the number of samples required to achieve a specified level of power]

---

## In a Nutshell

.center2[A ***power analysis*** is a calculation that can help you to determine a minimum sample size for your study]

---

## Four Components

.pull-left[
<br>
<p id="center" style="color:#74cbda; font-weight: bold; border:1px; border-style:solid; border-color:#74cbda; border-radius: 25px; padding: 0.3em;">
<i>Expected effect size</i><br><br>
standardized way of expressing the magnitude of the expected result of your study typically based on similar studies or a pilot study
</p>
]

.pull-right[
<br>
<p id="center" style="color:#bca0dc; font-weight: bold; border:1px; border-style:solid; border-color:#bca0dc; border-radius: 25px; padding: 0.3em;">
<i>Sample size</i><br><br>
the minimum number of observations needed to observe an effect of a certain size with a given power level
</p>
]

<br>
<br>
.pull-left[
<p id="center" style="color:#e69288; font-weight: bold; border:1px; border-style:solid; border-color:#e69288; border-radius: 25px; padding: 0.3em;">
<i>Significance level ( `\alpha` )</i><br><br>
the maximum risk of rejecting a true null hypothesis that you are willing to take - typically set at 5% or lower
</p>
]

.pull-right[
<p id="center" style="color:#8ec9bb; font-weight: bold; border:1px; border-style:solid; border-color:#8ec9bb; border-radius: 25px; padding: 0.3em;">
<i>(Statistical) Power</i><br><br>
the likelihood that a test will detect an effect of a certain size if there is one - typically set at 80% or higher
</p>
]

---

# A Rundown of Power Analysis in <svg aria-hidden="true" role="img" viewBox="0 0 581 512" style="height:1em;width:1.13em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:steelblue;overflow:visible;position:relative;"><path d="M581 226.6C581 119.1 450.9 32 290.5 32S0 119.1 0 226.6C0 322.4 103.3 402 239.4 418.1V480h99.1v-61.5c24.3-2.7 47.6-7.4 69.4-13.9L448 480h112l-67.4-113.7c54.5-35.4 88.4-84.9 88.4-139.7zm-466.8 14.5c0-73.5 98.9-133 220.8-133s211.9 40.7 211.9 133c0 50.1-26.5 85-70.3 106.4-2.4-1.6-4.7-2.9-6.4-3.7-10.2-5.2-27.8-10.5-27.8-10.5s86.6-6.4 86.6-92.7-90.6-87.9-90.6-87.9h-199V361c-74.1-21.5-125.2-67.1-125.2-119.9zm225.1 38.3v-55.6c57.8 0 87.8-6.8 87.8 27.3 0 36.5-38.2 28.3-87.8 28.3zm-.9 72.5H365c10.8 0 18.9 11.7 24 19.2-16.1 1.9-33 2.8-50.6 2.9v-22.1z"/></svg>

---

## Getting Ready

To follow along, please make sure to do the following

1. Open up a blank `.R` script

2. Run `install.packages("pwr", dependencies = TRUE)` in the Console

3. Load up the `pwr` package by running
    
    ```r
    library(pwr)
    ```

---

## List of Commands

<div class="datatables html-widget html-fill-item-overflow-hidden html-fill-item" id="htmlwidget-d773b834b83e14c62e8e" style="width:100%;height:auto;"></div>
<script type="application/json" data-for="htmlwidget-d773b834b83e14c62e8e">{"x":{"filter":"none","vertical":false,"extensions":["Buttons"],"data":[["<kbd>pwr.2p.test</kbd>","<kbd>pwr.2p2n.test</kbd>","<kbd>pwr.anova.test</kbd>","<kbd>pwr.chisq.test</kbd>","<kbd>pwr.f2.test</kbd>","<kbd>pwr.p.test</kbd>","<kbd>pwr.r.test</kbd>","<kbd>pwr.t.test</kbd>","<kbd>pwr.t2n.test</kbd>"],["Two proportions (equal <i>n</i>)","Two proportions (unequal <i>n</i>)","Balanced one-way ANOVA","Chi-square test","General linear model","Proportion (one sample)","Correlation","<i>t</i>-tests (one sample, two samples, paired)","<i>t</i>-test (two samples with unequal <i>n</i>)"]],"container":"<table class=\"FALSE\">\n  <thead>\n    <tr>\n      <th>Syntax<\/th>\n      <th>Description<\/th>\n    <\/tr>\n  <\/thead>\n<\/table>","options":{"pageLength":10,"searching":false,"autoWidth":false,"dom":"Brti","buttons":["copy","csv","excel","pdf"],"lengthMenu":false,"initComplete":"function(settings, json) {\n$(this.api().table().header()).css({'color': '#fff'});\n}","columnDefs":[],"order":[],"orderClasses":false,"rowCallback":"function(row, data, displayNum, displayIndex, dataIndex) {\nvar value=data[0]; $(this.api().cell(row, 0).node()).css({'color':'white','background-color':'#212121'});\nvar value=data[1]; $(this.api().cell(row, 1).node()).css({'color':'white','background-color':'#212121'});\n}"},"callback":"function(table) {\n$(\"button.buttons-copy\").css(\"color\",\"#bb86fc\"); \n                           $(\"button.buttons-csv\").css(\"color\",\"#bb86fc\"); \n                           $(\"button.buttons-excel\").css(\"color\",\"#bb86fc\"); \n                           $(\"button.buttons-pdf\").css(\"color\",\"#bb86fc\"); \n                            return table;\n}"},"evals":["options.initComplete","options.rowCallback","callback"],"jsHooks":[]}</script>

---

### *t*-test

---

#### <span style="color:#bff4ee">Equal Groups</span>

<center><kbd>pwr.t.test(n=, d=, sig.level=, power=, type=, alternative=)</kbd></center>

<br>
<br>
<table class="table table-condensed" style="width: auto !important; margin-left: auto; margin-right: auto;">
 <thead>
  <tr>
   <th style="text-align:left;background-color: #212121 !important;vertical-align: middle !important;"> Option </th>
   <th style="text-align:left;background-color: #212121 !important;vertical-align: middle !important;"> Description </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;width: 7em; background-color: #212121 !important;vertical-align: middle !important;"> <kbd>n</kbd> </td>
   <td style="text-align:left;width: 20em; background-color: #212121 !important;vertical-align: middle !important;"> Sample size </td>
  </tr>
  <tr>
   <td style="text-align:left;width: 7em; background-color: #212121 !important;vertical-align: middle !important;"> <kbd>d</kbd> </td>
   <td style="text-align:left;width: 20em; background-color: #212121 !important;vertical-align: middle !important;"> Power level </td>
  </tr>
  <tr>
   <td style="text-align:left;width: 7em; "> <kbd>sig.level</kbd> </td>
   <td style="text-align:left;width: 20em; "> Significance level (default of 0.05) </td>
  </tr>
  <tr>
   <td style="text-align:left;width: 7em; "> <kbd>type</kbd> </td>
   <td style="text-align:left;width: 20em; "> Choice of <i>t</i>-test - either <kbd>"two.sample"</kbd> (default), <kbd>"one.sample"</kbd>, or <kbd>"paired"</kbd> </td>
  </tr>
  <tr>
   <td style="text-align:left;width: 7em; "> <kbd>alternative</kbd> </td>
   <td style="text-align:left;width: 20em; "> Choice of direction - either <kbd>"two.sided"</kbd> (default), <kbd>"less"</kbd>, or <kbd>"greater"</kbd> </td>
  </tr>
</tbody>
</table>

---

```r
# Equal Groups

*pwr.t.test(d = 0.8,
*          sig.level = 0.05,
*          power = 0.8,
*          type = "two.sample",
*          alternative = "two.sided")
```
]
 
.panel2-sw1-auto[

```
## 
##      Two-sample t test power calculation 
## 
##               n = 25.52458
##               d = 0.8
##       sig.level = 0.05
##           power = 0.8
##     alternative = two.sided
## 
## NOTE: n is number in *each* group
```
]

---

#### <span style="color:#bff4ee">Unequal Groups</span>

<center><kbd>pwr.t2n.test(n1=, n2=, d=, sig.level=, power=, type=, alternative=)</kbd></center>

<br>
<br>
<table class="table table-condensed" style="width: auto !important; margin-left: auto; margin-right: auto;">
 <thead>
  <tr>
   <th style="text-align:left;background-color: #212121 !important;vertical-align: middle !important;"> Option </th>
   <th style="text-align:left;background-color: #212121 !important;vertical-align: middle !important;"> Description </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;width: 7em; background-color: #212121 !important;vertical-align: middle !important;"> <kbd>n1</kbd> </td>
   <td style="text-align:left;width: 20em; background-color: #212121 !important;vertical-align: middle !important;"> One sample size </td>
  </tr>
  <tr>
   <td style="text-align:left;width: 7em; background-color: #212121 !important;vertical-align: middle !important;"> <kbd>n2</kbd> </td>
   <td style="text-align:left;width: 20em; background-color: #212121 !important;vertical-align: middle !important;"> The other sample size </td>
  </tr>
  <tr>
   <td style="text-align:left;width: 7em; "> <kbd>d</kbd> </td>
   <td style="text-align:left;width: 20em; "> Power level </td>
  </tr>
  <tr>
   <td style="text-align:left;width: 7em; "> <kbd>sig.level</kbd> </td>
   <td style="text-align:left;width: 20em; "> Significance level (default of 0.05) </td>
  </tr>
  <tr>
   <td style="text-align:left;width: 7em; "> <kbd>type</kbd> </td>
   <td style="text-align:left;width: 20em; "> Choice of <i>t</i>-test - either <kbd>"two.sample"</kbd> (default), <kbd>"one.sample"</kbd>, or <kbd>"paired"</kbd> </td>
  </tr>
  <tr>
   <td style="text-align:left;width: 7em; "> <kbd>alternative</kbd> </td>
   <td style="text-align:left;width: 20em; "> Choice of direction - either <kbd>"two.sided"</kbd> (default), <kbd>"less"</kbd>, or <kbd>"greater"</kbd> </td>
  </tr>
</tbody>
</table>

---

```r
# Unequal Groups

*pwr.t2n.test(n1 = 28,
*            n2 = 35,
*            d = 0.5)
```
]
 
.panel2-sw2-auto[

```
## 
##      t test power calculation 
## 
##              n1 = 28
##              n2 = 35
##               d = 0.5
##       sig.level = 0.05
##           power = 0.4924588
##     alternative = two.sided
```
]

---

### ANOVA

<center><kbd>pwr.anova.test(k=, n=, f=, sig.level=, power=)</kbd></center>

<br>
<br>
<table class="table table-condensed" style="width: auto !important; margin-left: auto; margin-right: auto;">
 <thead>
  <tr>
   <th style="text-align:left;background-color: #212121 !important;vertical-align: middle !important;"> Option </th>
   <th style="text-align:left;background-color: #212121 !important;vertical-align: middle !important;"> Description </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;width: 7em; background-color: #212121 !important;vertical-align: middle !important;"> <kbd>k</kbd> </td>
   <td style="text-align:left;width: 20em; background-color: #212121 !important;vertical-align: middle !important;"> Number of groups </td>
  </tr>
  <tr>
   <td style="text-align:left;width: 7em; background-color: #212121 !important;vertical-align: middle !important;"> <kbd>n</kbd> </td>
   <td style="text-align:left;width: 20em; background-color: #212121 !important;vertical-align: middle !important;"> Common sample size in each group </td>
  </tr>
  <tr>
   <td style="text-align:left;width: 7em; "> <kbd>f</kbd> </td>
   <td style="text-align:left;width: 20em; "> Effect size </td>
  </tr>
  <tr>
   <td style="text-align:left;width: 7em; "> <kbd>sig.level</kbd> </td>
   <td style="text-align:left;width: 20em; "> Significance level (default of 0.05) </td>
  </tr>
  <tr>
   <td style="text-align:left;width: 7em; "> <kbd>power</kbd> </td>
   <td style="text-align:left;width: 20em; "> Power level </td>
  </tr>
</tbody>
</table>

---

```r
*pwr.anova.test(k = 3,
*              f = 0.25,
*              sig.level = 0.05,
*              power = 0.9)
```
]
 
.panel2-sw3-auto[

```
## 
##      Balanced one-way analysis of variance power calculation 
## 
##               k = 3
##               n = 68.49707
##               f = 0.25
##       sig.level = 0.05
##           power = 0.9
## 
## NOTE: n is number in each group
```
]

---

### Correlations

<center><kbd>pwr.r.test(n=, r=, sig.level=, power=, alternative=)</kbd></center>

<br>
<br>
<table class="table table-condensed" style="width: auto !important; margin-left: auto; margin-right: auto;">
 <thead>
  <tr>
   <th style="text-align:left;background-color: #212121 !important;vertical-align: middle !important;"> Option </th>
   <th style="text-align:left;background-color: #212121 !important;vertical-align: middle !important;"> Description </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;width: 7em; background-color: #212121 !important;vertical-align: middle !important;"> <kbd>n</kbd> </td>
   <td style="text-align:left;width: 20em; background-color: #212121 !important;vertical-align: middle !important;"> Sample size </td>
  </tr>
  <tr>
   <td style="text-align:left;width: 7em; background-color: #212121 !important;vertical-align: middle !important;"> <kbd>r</kbd> </td>
   <td style="text-align:left;width: 20em; background-color: #212121 !important;vertical-align: middle !important;"> Effect size </td>
  </tr>
  <tr>
   <td style="text-align:left;width: 7em; "> <kbd>sig.level</kbd> </td>
   <td style="text-align:left;width: 20em; "> Significance level (default of 0.05) </td>
  </tr>
  <tr>
   <td style="text-align:left;width: 7em; "> <kbd>power</kbd> </td>
   <td style="text-align:left;width: 20em; "> Power level </td>
  </tr>
  <tr>
   <td style="text-align:left;width: 7em; "> <kbd>alternative</kbd> </td>
   <td style="text-align:left;width: 20em; "> Choice of direction - either <kbd>"two.sided"</kbd> (default), <kbd>"less"</kbd>, or <kbd>"greater"</kbd> </td>
  </tr>
</tbody>
</table>

---

```r
*pwr.r.test(r = 0.40,
*          sig.level = 0.05,
*          power = 0.80,
*          alternative = "greater")
```
]
 
.panel2-sw4-auto[

```
## 
##      approximate correlation power calculation (arctangh transformation) 
## 
##               n = 36.50995
##               r = 0.4
##       sig.level = 0.05
##           power = 0.8
##     alternative = greater
```
]

---

### General Linear Models

<center><kbd>pwr.f2.test(u=, v=, f2=, sig.level=, power=) </kbd></center>

<br>
<br>
<table class="table table-condensed" style="width: auto !important; margin-left: auto; margin-right: auto;">
 <thead>
  <tr>
   <th style="text-align:left;background-color: #212121 !important;vertical-align: middle !important;"> Option </th>
   <th style="text-align:left;background-color: #212121 !important;vertical-align: middle !important;"> Description </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;width: 7em; background-color: #212121 !important;vertical-align: middle !important;"> <kbd>u</kbd> </td>
   <td style="text-align:left;width: 20em; background-color: #212121 !important;vertical-align: middle !important;"> Numerator of degrees of freedom (df) </td>
  </tr>
  <tr>
   <td style="text-align:left;width: 7em; background-color: #212121 !important;vertical-align: middle !important;"> <kbd>v</kbd> </td>
   <td style="text-align:left;width: 20em; background-color: #212121 !important;vertical-align: middle !important;"> Denominator of degrees of freedom (df) </td>
  </tr>
  <tr>
   <td style="text-align:left;width: 7em; "> <kbd>f2</kbd> </td>
   <td style="text-align:left;width: 20em; "> Effect size </td>
  </tr>
  <tr>
   <td style="text-align:left;width: 7em; "> <kbd>sig.level</kbd> </td>
   <td style="text-align:left;width: 20em; "> Significance level (default of 0.05) </td>
  </tr>
  <tr>
   <td style="text-align:left;width: 7em; "> <kbd>power</kbd> </td>
   <td style="text-align:left;width: 20em; "> Power level </td>
  </tr>
</tbody>
</table>

---

```r
*pwr.f2.test(u = 2,
*           f2 = 0.3/(1 - 0.3),
*           sig.level = 0.001,
*           power = 0.8)
```
]
 
.panel2-sw5-auto[

```
## 
##      Multiple regression power calculation 
## 
##               u = 2
##               v = 49.88971
##              f2 = 0.4285714
##       sig.level = 0.001
##           power = 0.8
```
]

---

### Tests of Proportions

---

#### <span style="color:#bff4ee">Equal Groups</span>

<center><kbd>pwr.2p.test(h=, n=, sig.level=, power=)</kbd></center>

<br>
<br>
<table class="table table-condensed" style="width: auto !important; margin-left: auto; margin-right: auto;">
 <thead>
  <tr>
   <th style="text-align:left;background-color: #212121 !important;vertical-align: middle !important;"> Option </th>
   <th style="text-align:left;background-color: #212121 !important;vertical-align: middle !important;"> Description </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;width: 7em; background-color: #212121 !important;vertical-align: middle !important;"> <kbd>h</kbd> </td>
   <td style="text-align:left;width: 20em; background-color: #212121 !important;vertical-align: middle !important;"> Effect size </td>
  </tr>
  <tr>
   <td style="text-align:left;width: 7em; background-color: #212121 !important;vertical-align: middle !important;"> <kbd>n</kbd> </td>
   <td style="text-align:left;width: 20em; background-color: #212121 !important;vertical-align: middle !important;"> Sample size </td>
  </tr>
  <tr>
   <td style="text-align:left;width: 7em; "> <kbd>sig.level</kbd> </td>
   <td style="text-align:left;width: 20em; "> Significance level (default of 0.05) </td>
  </tr>
  <tr>
   <td style="text-align:left;width: 7em; "> <kbd>power</kbd> </td>
   <td style="text-align:left;width: 20em; "> Power level </td>
  </tr>
</tbody>
</table>

---

```r
# Equal Groups

*pwr.2p.test(h = ES.h(p1 = 0.55, p2 = 0.50),
*           sig.level = 0.05,
*           power = 0.80)
```
]
 
.panel2-sw6-auto[

```
## 
##      Difference of proportion power calculation for binomial distribution (arcsine transformation) 
## 
##               h = 0.1001674
##               n = 1564.529
##       sig.level = 0.05
##           power = 0.8
##     alternative = two.sided
## 
## NOTE: same sample sizes
```
]

---

#### <span style="color:#bff4ee">Unequal Groups</span>

<center><kbd>pwr.2p2n.test(h =, n1 =, n2 =, sig.level=, power=)</kbd></center>

<br>
<br>
<table class="table table-condensed" style="width: auto !important; margin-left: auto; margin-right: auto;">
 <thead>
  <tr>
   <th style="text-align:left;background-color: #212121 !important;vertical-align: middle !important;"> Option </th>
   <th style="text-align:left;background-color: #212121 !important;vertical-align: middle !important;"> Description </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;width: 7em; background-color: #212121 !important;vertical-align: middle !important;"> <kbd>h</kbd> </td>
   <td style="text-align:left;width: 20em; background-color: #212121 !important;vertical-align: middle !important;"> Effect size </td>
  </tr>
  <tr>
   <td style="text-align:left;width: 7em; background-color: #212121 !important;vertical-align: middle !important;"> <kbd>n1</kbd> </td>
   <td style="text-align:left;width: 20em; background-color: #212121 !important;vertical-align: middle !important;"> One sample size </td>
  </tr>
  <tr>
   <td style="text-align:left;width: 7em; "> <kbd>n2</kbd> </td>
   <td style="text-align:left;width: 20em; "> The other sample size </td>
  </tr>
  <tr>
   <td style="text-align:left;width: 7em; "> <kbd>sig.level</kbd> </td>
   <td style="text-align:left;width: 20em; "> Significance level (default of 0.05) </td>
  </tr>
  <tr>
   <td style="text-align:left;width: 7em; "> <kbd>power</kbd> </td>
   <td style="text-align:left;width: 20em; "> Power level </td>
  </tr>
  <tr>
   <td style="text-align:left;width: 7em; "> <kbd>alternative</kbd> </td>
   <td style="text-align:left;width: 20em; "> Choice of direction - either <kbd>"two.sided"</kbd> (default), <kbd>"less"</kbd>, or <kbd>"greater"</kbd> </td>
  </tr>
</tbody>
</table>

---

```r
# Unequal Groups

*pwr.2p2n.test(h = 0.2,
*             n1 = 763,
*             power = 0.8,
*             sig.level = 0.05,
*             alternative = "greater")
```
]
 
.panel2-sw7-auto[

```
## 
##      difference of proportion power calculation for binomial distribution (arcsine transformation) 
## 
##               h = 0.2
##              n1 = 763
##              n2 = 193.8285
##       sig.level = 0.05
##           power = 0.8
##     alternative = greater
## 
## NOTE: different sample sizes
```
]

---

### Chi-square

<center><kbd>pwr.chisq.test(w =, N =, df =, sig.level =, power = )</kbd></center>

<br>
<br>
<table class="table table-condensed" style="width: auto !important; margin-left: auto; margin-right: auto;">
 <thead>
  <tr>
   <th style="text-align:left;background-color: #212121 !important;vertical-align: middle !important;"> Option </th>
   <th style="text-align:left;background-color: #212121 !important;vertical-align: middle !important;"> Description </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;width: 7em; background-color: #212121 !important;vertical-align: middle !important;"> <kbd>w</kbd> </td>
   <td style="text-align:left;width: 20em; background-color: #212121 !important;vertical-align: middle !important;"> Effect size </td>
  </tr>
  <tr>
   <td style="text-align:left;width: 7em; background-color: #212121 !important;vertical-align: middle !important;"> <kbd>N</kbd> </td>
   <td style="text-align:left;width: 20em; background-color: #212121 !important;vertical-align: middle !important;"> Total sample size </td>
  </tr>
  <tr>
   <td style="text-align:left;width: 7em; "> <kbd>df</kbd> </td>
   <td style="text-align:left;width: 20em; "> Degrees of freedom </td>
  </tr>
  <tr>
   <td style="text-align:left;width: 7em; "> <kbd>sig.level</kbd> </td>
   <td style="text-align:left;width: 20em; "> Significance level (default of 0.05) </td>
  </tr>
  <tr>
   <td style="text-align:left;width: 7em; "> <kbd>power</kbd> </td>
   <td style="text-align:left;width: 20em; "> Power level </td>
  </tr>
</tbody>
</table>

---

```r
*pwr.chisq.test(w = 0.1,
*              power = 0.9,
*              df = 1,
*              sig.level = 0.01)
```
]
 
.panel2-sw8-auto[

```
## 
##      Chi squared power calculation 
## 
##               w = 0.1
##               N = 1487.939
##              df = 1
##       sig.level = 0.01
##           power = 0.9
## 
## NOTE: N is the number of observations
```
]

---

# Power Analysis Plots

.center2[**Power curves** are line plots that show how the change in variables, such as effect size and sample size, impact the power of the statistical test]

.footnote[Unfortunately we do not have the bandwidth to cover power curves here, but an example and annotated script is provided on the next slide should you be interested]

---

## Example

```
## Warning: The `size` argument of `element_line()` is deprecated as of ggplot2 3.4.0.
## ℹ Please use the `linewidth` argument instead.
```

.footnote[
<div class='footsbs'>
<a href="scripts/powercurves_proptest.R.zip" target='_blank' download="Powercurves Example script">
<img src="img/Rscript-ico.png" alt="Rscript icon" width='36'>
</a>
</div>
]
    
---

# Note: Choosing a Starting Effect Size

> Estimating an expected effect size is the most difficult parameter to determine

> It requires that you have experience with the content and context of a study and any corresponding measures (e.g. using previous studies and results to calculate possible effect sizes )

> If you don't have access to this information or if a study is completely novel, Cohen (1988) created some basic estimations and benchmarks. Much like other guidelines such as *p*-values (ugh), Cohen's Kappa, etc, these serve as a starting point and by no means should they be treated as static rules!

<table class="table table-condensed" style="width: auto !important; margin-left: auto; margin-right: auto;">
 <thead>
  <tr>
   <th style="text-align:left;"> Approach </th>
   <th style="text-align:center;"> Variable </th>
   <th style="text-align:center;"> Small Effect </th>
   <th style="text-align:center;"> Medium Effect </th>
   <th style="text-align:center;"> Large Effect </th>
  </tr>
 </thead>
<tbody>
  <tr>
   <td style="text-align:left;width: 10em; "> <i>t</i>-test </td>
   <td style="text-align:center;width: 7em; "> <kbd>d</kbd> </td>
   <td style="text-align:center;width: 7em; "> 0.20 </td>
   <td style="text-align:center;width: 7em; "> 0.50 </td>
   <td style="text-align:center;width: 7em; "> 0.80 </td>
  </tr>
  <tr>
   <td style="text-align:left;width: 10em; "> ANOVA </td>
   <td style="text-align:center;width: 7em; "> <kbd>f</kbd> </td>
   <td style="text-align:center;width: 7em; "> 0.10 </td>
   <td style="text-align:center;width: 7em; "> 0.25 </td>
   <td style="text-align:center;width: 7em; "> 0.40 </td>
  </tr>
  <tr>
   <td style="text-align:left;width: 10em; "> General Linear Models </td>
   <td style="text-align:center;width: 7em; "> <kbd>f2</kbd> </td>
   <td style="text-align:center;width: 7em; "> 0.02 </td>
   <td style="text-align:center;width: 7em; "> 0.15 </td>
   <td style="text-align:center;width: 7em; "> 0.35 </td>
  </tr>
  <tr>
   <td style="text-align:left;width: 10em; "> Tests of Proportions </td>
   <td style="text-align:center;width: 7em; "> <kbd>h</kbd> </td>
   <td style="text-align:center;width: 7em; "> 0.20 </td>
   <td style="text-align:center;width: 7em; "> 0.50 </td>
   <td style="text-align:center;width: 7em; "> 0.80 </td>
  </tr>
  <tr>
   <td style="text-align:left;width: 10em; "> Chi-square </td>
   <td style="text-align:center;width: 7em; "> <kbd>w</kbd> </td>
   <td style="text-align:center;width: 7em; "> 0.10 </td>
   <td style="text-align:center;width: 7em; "> 0.30 </td>
   <td style="text-align:center;width: 7em; "> 0.50 </td>
  </tr>
</tbody>
</table>

.footnote[
<div class='footsbs'>
<a href="files/Cohen%20(1988).pdf" target='_blank' download="Statistical Power Analysis for the Behavioral Sciences pdf">
<img src="img/pdf-ico.png" alt="PDF icon" width='36'>
</a>
</div>
]

---

# Terminology is Important!

.pull-right[
<blockquote class="twitter-tweet" data-width="550" data-lang="en" data-dnt="true" data-theme="light"><p lang="en" dir="ltr">Please stop saying studies are underpowered without clarifying *what* the design is underpowered for. It’s a vague descriptor that allows critique without any concrete reasoning behind it. Studies aren’t underpowered. Designs are underpowered *for some effects in some tests*. ></p>— Richard D. Morey (@richarddmorey@tech.lgbt) (@richarddmorey) <a href="https://twitter.com/richarddmorey/status/1076204872091893760?ref_src=twsrc%5Etfw">December 21, 2018</a></blockquote>

]

<br>
.pull-left[
For more on this, click the graphic to the right
]

.pull-right[
<a href="https://towardsdatascience.com/why-you-shouldnt-say-this-study-is-underpowered-627f002ddf35">
<img src="img/tds.png" width="50%" />
</a>
]

---

## Additional Resources

If you would like a different view of power and its use, please take a look at the videos below
<br>
<br>

<span style="color:#8ec9bb; font-weight: bold; font-style: italic;">(Statistical) Power</span>
</center>
<p align="center">
    <iframe width="450" height="252" src="https://www.youtube.com/embed/Rsc5znwR5FA" title="Statistical Power, Clearly Explained!!!" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>
</p>
]

***Power analysis***
</center>
<p align="center">
    <iframe width="450" height="252" src="https://www.youtube.com/embed/VX_M3tIyiYk" title="Power Analysis, Clearly Explained!!!" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>
</p>
]

---

## Thats it!

If you have any questions, please reach out

<center>
<a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/"><img alt="Creative Commons License" style="border-width:0" src="https://i.creativecommons.org/l/by-nc-sa/4.0/88x31.png" /></a><br /><br />This work is licensed under a <br /><a rel="license" href="http://creativecommons.org/licenses/by-nc-sa/4.0/">Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License</a>
</center>