==Running Tutorials on Your Computer==
Since Starting Fall 2019 we adopted adopt [https://www.docker.com/ docker] to run our course material . We have created various [https://hub.docker.com/u/statisticalgenetics docker repositories] with source material freely available from [https://github.com/statgenetics/statgen-courses github] for users to readily setup and reproduce our tutorials on their own computers. These docker images can also be used as production tool to run relevant software on your computer (Mac, Linux or Windows) or even a high performance computing cluster (if properly configured) for your own data analysis.
In this document we will focus on discussing how ===General instructions===* [https://github.com/statgenetics/statgen-courses/wiki/How-to set it up and run -launch-course -tutorials #alternative-to-cloud-server-use-your-own-computer Instructions to setup course tutorial environment on your computer, using these docker images and optionally a utility script we created ]* [https://github.com/statgenetics/statgen-courses/wiki/How-to streamline various docker commands-launch-course-tutorials#option-1-launch-exercise-in-jupyterlab Instructions to run course tutorial through JupyterLab]* [https://github.com/statgenetics/statgen-courses/wiki/How-to-launch-course-tutorials#option-2-launch-from-command-shell Instructions to run course tutorial through command line terminal]
===Pre-requisites===
Software you need to install on your computer are <code>SoS</code> (a workflow system to run our course utility script) and <code>docker</code>.
====Mac and Linux users====
<code>SoS</code> requires Python 3.6+ to run. It is recommended that you install [https://docs.conda.io/en/latest/miniconda.html Miniconda] to run Python 3 if you don't have it already. Once you have Python 3 installed, simply type <code>pip install sos</code> to install <code>SoS</code>, or, check out [https://vatlab.github.io/sos-docs/running.html here for alternative installation methods] if you have troubles with that command. To install <code>docker</code> from command line please follow our instructions [http://statgen.us/lab-wiki/orientation/jupyter-setup.html#install-docker here]. Alternatively Mac users can download docker app for Mac and install from a graphical interface.
Finally please download our utility script [https://raw.githubusercontent.com/statgenetics/statgen-courses/master/src/statgen-setup <code>src/statgen-setup</code>] to your <code>PATH</code> and change it to executable, eg, <code>chmod +x ~/bin/statgen-setup</code> if you put it under <code>~/bin</code> which is part of your <code>PATH</code>. To verify your setup, type:
<pre>statgen-setup -h</pre>
you should see some meaningful output.
==== Windows users====
Running these tutorials in Windows is currently not supported. Although in principle these docker images will also work in Windows, this has not yet been tested out and we are unable to provide sure instructions to setting it up. The utility script "statgen-setup" that we provide will certainly need adjustments (though minor) to work with Windows.
===Tutorial specific instructions===
We use a script [https://github.com/statgenetics/statgen-courses/blob/master/src/statgen-setup "statgen-setup"] to start the docker based environments for these tutorials. Please refer to the previous section for instructions on the installation of this script.
Material and instructions for specific exercise are listed in each section below (''only those using statgen-setup command are relevant to our docker based tutorials''). They provide links to materials and a minimal set of commands to use for launching and running an exercise. For advanced options Command "statgen-setup login" will allow you to start and other features provided by our utility script please read our [https://githublogin to a Linux command line environment (regardless of your current computational environment) to perform all analysis in command shells.com/statgenetics/This works for all the tutorials on this page. Additionally, some tutorials support command "statgen-courses/blob/master/README.md complete documentation setup launch" which will start a JupyterLab server to perform the utility script here]analysis.
==Alohomora==
* [http://gmc.mdc-berlin.de/alohomora/ Software Link]
==AnnotationAnnovar complex traits==* [https://github.com/statgenetics/statgen-courses/blob/master/handout/FunctionalAnnotation.2019.docx pdf Functional Annotation Exercise <nowiki>[DOCXPDF]</nowiki>]
* [[Commands in Annotation Exercise|Exercise Commands]]
</pre>
The "statgen-setup" script is available [https://github.com/statgenetics/statgen-courses/blob/master/src/statgen-setup here] and can be installed following [https://github.com/statgenetics/statgen-courses/blob/master/README.md#prepare-your-computer-to-manage-the-tutorials these instructions]. ==Annovar MEndelianMendelian traits==
* [http://statgen.us/files/tutorials/FunctionalAnnotation_Annovar_final.pdf Exercise <nowiki>[PDF]</nowiki>]
* [https://statgen.research.bcm.eduus/files/2017/09/commands/annovar-functional_annotation.txt Commands Part I - Functional Annotation]* [https://statgen.research.bcm.eduus/files/2017/09/commands/annovar-variant_filtering.txt Commands Part II - Variant Filtering]
==Cochran Armitage Trend Test==
* [httphttps://statgengithub.uscom/statgenetics/statgen-courses/blob/filesmaster/tutorialshandout/Armitage_sample_size_revisedArmitage_sample_size.pdf CATT Exercise <nowiki>[PDF]</nowiki>]
==GATK and IGV==
* [http://statgen.us/files/tutorials/Variant_calling_exercices_GATK_IGV.pdf GATK/IGV Exercise <nowiki>[PDF]</nowiki>]
====Software Links====
:* [https://www.broadinstitute.org/gatk/ GATK]
:* [http://www.broadinstitute.org/igv/ IGV] ([http://statgen.us/files/software/igv.jar Download JAR file])
====Result Files==Epistasis (PLINK and CASSI)==:* [httphttps://statgengithub.uscom/filesstatgenetics/2016statgen-courses/07blob/gatk_resultsmaster/handout/Epistasis_PLINK_CASSI.tar.gz GATKpdf Epistasis exercise <nowiki>[PDF]</IGV exercisenowiki>]
To run the exercise from docker image provided, <pre>statgen-setup login --tutorial epistasis</pre> ==GeminiFastLMM==* [https://github.com/statgenetics/statgen-courses/blob/master/handout/GeminiFASTLMM.docx Gemini pdf FastLMM exercise <nowiki>[DOCXPDF]</nowiki>]
To run the exercise from docker image provided,
<pre>statgen-setup login --tutorial geminifastlmm-gcta
</pre>
The "statgen-setup" script is available [https://github.com/statgenetics/statgen-courses/blob/master/src/statgen-setup here] and can be installed following [https://github.com/statgenetics/statgen-courses/blob/master/README.md#prepare-your-computer-to-manage-the-tutorials these instructions].
==GenABELFine-mapping (SuSiE method)==* [httphttps://statgengithub.uscom/filesstatgenetics/tutorialsstatgen-courses/blob/master/genabelhandout/GenABEL_with_PLINK1finemapping.9docx susieR Exercise <nowiki>[DOCX]</nowiki>]* [https://github.com/statgenetics/statgen-courses/blob/master/handout/finemapping_answers.pdf GenABEL docx susieR Exercise Answers <nowiki>[PDFDOCX]</nowiki>]* [https://github.com/statgenetics/statgen-courses/blob/master/notebooks/finemapping.ipynb susieR Exercise <nowiki>[GeneABEL Ipython notebook]</nowiki>]* [https://github.com/statgenetics/statgen-courses/blob/master/notebooks/finemapping_answers.ipynb susieR Exercise|Exercise CommandsAnswers <nowiki>[Ipython notebook]</nowiki>]
To install from packages, follow the configuration steps above and run the following command.
To run the exercise from docker image provided, <pre>
sudo aptstatgen-setup launch -
get install genabel-tutorial
finemap</pre>
The ==GCTA==* [https://github.com/statgenetics/statgen-courses/blob/master/handout/GCTA.pdf GCTA exercise
's files will then be installed in <nowiki>[PDF]</nowiki>] To run the
folder ''exercise from docker image provided, <pre>statgen-setup login --tutorial fastlmm-gcta</
homepre> ==Gemini==* [https:/
shared/
genabel''github.
You can com/statgenetics/statgen-courses/blob/master/handout/Gemini.docx Gemini exercise <nowiki>[DOCX]</nowiki>] To run
from there or copy the files into your user's home directory and proceed with the exercise
.from docker image provided, <pre>statgen-setup login --tutorial gemini</pre>
==Genehunter==
==GWAS: Data Quality Control==
* [httphttps://statgengithub.uscom/filesstatgenetics/tutorials/gwasstatgen-qccourses/PLINK_data_QC_V2blob/master/handout/PLINK_data_QC.pdf Exercise <nowiki>[PDF]</nowiki>]* [https://github.com/statgenetics/statgen-courses/blob/master/notebooks/PLINK_Data_QC.ipynb Exercise <nowiki>[IPython Notebook]</nowiki>]
* [[GWAS Data QC Exercise|Exercise Commands]]
====Installing Packages====To install run the exercise via JupyterLab from packagesdocker image provided, follow the configuration steps above and run the following command.
<pre>sudo aptstatgen-setup launch -get install plink-tutorialplink</pre>The exercise's files will then be installed in the folder ''/home/shared/Plink_DataQC''. You Alternatively, you can opt to run from there or copy the files into your user's home directory and proceed with the exercise.via command terminal from docker image provided, <pre>statgen-setup login --tutorial plink</pre>
==GWAS: Association Analysis Controlling for Population Substructure==
* [httphttps://statgengithub.uscom/filesstatgenetics/tutorials/gwasstatgen-substructurecourses/GWAS_Data_Controlling_for_Substructure_V2blob/master/handout/PLINK_Substructure.pdf Exercise <nowiki>[PDF]</nowiki>]* [https://github.com/statgenetics/statgen-courses/blob/master/notebooks/PLINK_Substructure.ipynb Exercise <nowiki>[IPython Notebook]</nowiki>]
* [[GWAS_Controlling_for_Population_Substructure|Exercise Commands]]
====Installing Packages====To install run the exercise via JupyterLab from packagesdocker image provided, follow the configuration steps above and run the following command.
<pre>sudo aptstatgen-setup launch -get install plink-tutorialplink</pre>The exercise's files will then be installed in the folder ''/home/shared/Plink_substructure''. You can run from there or copy the files into your user's home directory and proceed with the exercise.
==Homozygosity Mapper==Alternatively, you can opt to run the exercise via command terminal from docker image provided,
<pre>statgen-setup login --tutorial plink
</pre>
==Homozygosity Mapper==
* [http://statgen.us/files/tutorials/Homozygosity_mapping_exercise_part_1.pdf Exercise Part I]
* [http://statgen.us/files/tutorials/Homozygosity_mapping_exercise_part_2.pdf Exercise Part II]
==IGV==
* [http://statgen.us/files/software/igv/igv-2.3.97.jar igv.jar Software Link]
* [https://github.com/statgenetics/statgen-courses/blob/master/handout/IGV.docx IGV exercise <nowiki>[DOCX]</nowiki>]
* [http://statgen.us/files/igv_exercise.zip Exercise files (VCF and BAM)]
To run the exercise from docker image provided,
<pre>statgen-setup login --tutorial igv
</pre>
The "statgen-setup" script is available [https://github.com/statgenetics/statgen-courses/blob/master/src/statgen-setup here] and can be installed following [https://github.com/statgenetics/statgen-courses/blob/master/README.md#prepare-your-computer-to-manage-the-tutorials these instructions].
==Linkage/FastLinkage==
</pre>
The "==LD clumping==* [https://github.com/cumc/bioworkflows/blob/master/GWAS/LD_Clumping.ipynb LD Clumping Exercise <nowiki>[IPython Notebook]</nowiki>] To run the exercise from docker image provided, <pre>statgen-setup" script is available launch --tutorial clumping</pre> ==MR-JTI for TWAS==* [https://github.com/statgenetics/statgen-courses/blob/master/srcnotebooks/MR_JTI.ipynb MR-JTI Exercise <nowiki>[IPython Notebook]</nowiki>] '''Under Development''' To run the exercise from docker image provided, <pre>statgen-setup herelogin --tutorial twas</pre> ==Pleiotropy==* [https://github.com/statgenetics/statgen-courses/blob/master/handout/Pleiotropy.pdf Pleiotropy Exercise <nowiki>[PDF] and can be installed following </nowiki>]* [https://github.com/statgenetics/statgen-courses/blob/master/READMEhandout/Pleiotropy_answers.md#preparepdf Pleiotropy Answers to Questions <nowiki>[PDF]</nowiki>] To run the exercise from docker image provided, <pre>statgen-yoursetup login -computer-to-managetutorial pleiotropy</pre> ==Polygenic risk prediction (NPS method)==* [https://github.com/statgenetics/statgen-courses/blob/master/handout/NPS.pdf PRS NPS Exercise <nowiki>[DOCX]</nowiki>] To run theexercise from docker image provided, <pre>statgen-tutorials these instructionssetup login --tutorial nps</pre> ==Polygenic risk prediction (LDpred2 method)==* [https://github.com/statgenetics/statgen-courses/blob/master/handout/ldpred2_example.pdf PRS LDpred2 Exercise <nowiki>[PDF]</nowiki>]* [https://github.com/cumc/bioworkflows/blob/master/ldpred/ldpred2_example.ipynb PRS LDpred2 Exercise <nowiki>[Ipython Notebook]</nowiki>] To run the exercise from docker image provided, <pre>statgen-setup launch --tutorial ldpred2</pre> Then follow prompts on the terminal output to open up the JupyterLab server in your web browser. If it is the first time you start this server, please open a command terminal inside JupyterLab, and type <pre>get-data</pre> to load the data-set to the JupyterLab workspace.
<!--
</pre>
The "statgen-setup" script is available [https://github.com/statgenetics/statgen-courses/blob/master/src/statgen-setup here] and can be installed following [https://github.com/statgenetics/statgen-courses/blob/master/README.md#prepare-your-computer-to-manage-the-tutorials these instructions].
==PSEQ==
* [https://github.com/statgenetics/statgen-courses/blob/master/handout/PSEQ.doc pdf PSEQ Exercise <nowiki>[DOCXPDF]</nowiki>]* [[https://github.com/statgenetics/statgen-courses/blob/master/notebooks/PSEQ.ipynb PSEQ Commands in Exercise|Exercise Commands<nowiki>[Ipython Notebook]</nowiki>]
To run the exercise from docker image provided,
<pre>statgen-setup launch --tutorial pseq
</pre>
Notice that since PSEQ exercise does not involve generating and visualizing plots, it is also fine to use a command terminal, instead of the JupyterLab server, to run this exercise and reproduce exactly what was described in the tutorial. To do so,
<pre>statgen-setup login --tutorial pseq
</pre>
The " ==R and PLINK== To run the exercise from docker image provided, <pre>statgen-setup" script is available launch --tutorial plink-r-nothnagel</pre> ==REGENIE==* [https://github.com/statgenetics/statgen-courses/blob/master/srchandout/statgen-setup hereregenie_example.pdf REGENIE Exercise <nowiki>[PDF] and can be installed following </nowiki>]* [https://github.com/statgenetics/statgen-courses/blob/master/READMEnotebooks/regenie_example.md#prepareipynb REGENIE Exercise <nowiki>[Ipython Notebook]</nowiki>] To run the exercise from docker image provided, <pre>statgen-yoursetup launch -computer-tutorial regenie</pre> Then follow prompts on the terminal output toopen up the JupyterLab server in your web browser. If it is the first time you start this server, please open a command terminal inside JupyterLab, and type <pre>get-manage-data</pre> to load thedata-tutorials these instructions]set to the JupyterLab workspace.
==Regression==
</pre>
The "statgen-setup" script is available [https://github.com/statgenetics/statgen-courses/blob/master/src/statgen-setup here] and can be installed following [https://github.com/statgenetics/statgen-courses/blob/master/README.md#prepare-your-computer-to-manage-the-tutorials these instructions].
==RV-TDT==
* [httphttps://statgengithub.uscom/statgenetics/statgen-courses/filesblob/tutorialsmaster/rvtdthandout/rvtdt_exercise_v3RV-TDT.pdf doc Exercise <nowiki>[PDFDOC]</nowiki>]
* [[RV-TDT|Exercise Commands]]
To run the exercise from docker image provided,
<pre>statgen-setup login --tutorial slink</pre> The "statgen-setup" script is available [https://github.com/statgenetics/statgen-courses/blob/master/src/statgen-setup here] and can be installed following [https://github.com/statgenetics/statgen-courses/blob/master/README.md#prepare-your-computer-to-manage-the-tutorials these instructions].
==SUPERLINK==
==Variant Association Tools==
* [https://statgenetics.github.iocom/statgenetics/statgen-courses/notebooksblob/master/handout/VAT.html docx VAT Exercise <nowiki>[HTMLDOCX]</nowiki>]* [[https://github.com/statgenetics/statgen-courses/blob/master/notebooks/VAT.ipynb VAT Commands in Exercise|Exercise Commands<nowiki>[Ipython notebook]</nowiki>]
To run the exercise from docker image provided,
<pre>statgen-setup login launch --tutorial vat
</pre>
The "statgen-setup" script is available [https://github.com/statgenetics/statgen-courses/blob/master/src/statgen-setup here] and can be installed following [https://github.com/statgenetics/statgen-courses/blob/master/README.md#prepare-your-computer-Then follow the prompts on the terminal output to-manage-open up the-tutorials these instructions]JupyterLab server in your web browser. ==VAT Mendelian==* [http://statgen.us/files/tutorials/VAT_Mendelian.pdf Exercise <nowiki>[PDF]</nowiki>]* [[VAT mendelian commands|Exercise Commands]] To install from packages, follow You should find the configuration steps above and run the following command. <pre>sudo apt-get install vtools-mendelian-tutorial</pre>The exercise's files will then be installed notebook in the folder ''/home/shared/vat''. You side panel, and you can run from there or copy the files into your user's home directory and proceed with the exerciseclick to open it.
