Difference between revisions of "Tutorials"
From Statistical Genetics Courses
Serveradmin (Talk | contribs) (→Running Tutorials on Your Computer) |
Serveradmin (Talk | contribs) (→Preparing Your Computer) |
||
| (39 intermediate revisions by the same user not shown) | |||
| Line 2: | Line 2: | ||
Starting Fall 2019 we adopt [https://www.docker.com/ docker] to run our course material . We have created various [https://hub.docker.com/u/statisticalgenetics docker repositories] with source material freely available from [https://github.com/statgenetics/statgen-courses github] for users to readily setup and reproduce our tutorials on their own computers. These docker images can also be used as production tool to run relevant software on your computer (Mac, Linux or Windows) or even a high performance computing cluster (if properly configured) for your own data analysis. | Starting Fall 2019 we adopt [https://www.docker.com/ docker] to run our course material . We have created various [https://hub.docker.com/u/statisticalgenetics docker repositories] with source material freely available from [https://github.com/statgenetics/statgen-courses github] for users to readily setup and reproduce our tutorials on their own computers. These docker images can also be used as production tool to run relevant software on your computer (Mac, Linux or Windows) or even a high performance computing cluster (if properly configured) for your own data analysis. | ||
| − | + | ===General instructions=== | |
* [https://github.com/statgenetics/statgen-courses/wiki/How-to-launch-course-tutorials#alternative-to-cloud-server-use-your-own-computer Instructions to setup course tutorial environment on your computer] | * [https://github.com/statgenetics/statgen-courses/wiki/How-to-launch-course-tutorials#alternative-to-cloud-server-use-your-own-computer Instructions to setup course tutorial environment on your computer] | ||
| − | * | + | * [https://github.com/statgenetics/statgen-courses/wiki/How-to-launch-course-tutorials#option-1-launch-exercise-in-jupyterlab Instructions to run course tutorial through JupyterLab] |
| − | * | + | * [https://github.com/statgenetics/statgen-courses/wiki/How-to-launch-course-tutorials#option-2-launch-from-command-shell Instructions to run course tutorial through command line terminal] |
| + | ===Preparing Your Computer=== | ||
| + | {| class="wikitable" | ||
| + | |- | ||
| + | ||{{#ev:youtube|3K-sGzxsyK0|240|center|Linux}} | ||
| + | ||{{#ev:youtube|DRCDNBlxZ-w|240|center|Mac}} | ||
| + | ||{{#ev:youtube|sxv45NCSFMk|240|center|Windows}} | ||
| + | |} | ||
| + | |||
| + | ===Running Exercises=== | ||
| + | {{#ev:youtube|OgHvRVtIIog|320}} | ||
===Tutorial specific instructions=== | ===Tutorial specific instructions=== | ||
| − | Material and instructions for specific exercise are listed in each section below (''only those using statgen-setup command are relevant to our docker based tutorials''). They provide links to materials and a minimal set of commands to use for launching and running an exercise. | + | We use a script [https://github.com/statgenetics/statgen-courses/blob/master/src/statgen-setup "statgen-setup"] to start the docker based environments for these tutorials. Please refer to the previous section for instructions on the installation of this script. |
| + | |||
| + | Material and instructions for specific exercise are listed in each section below (''only those using statgen-setup command are relevant to our docker based tutorials''). They provide links to materials and a minimal set of commands to use for launching and running an exercise. Command "statgen-setup login" will allow you to start and login to a Linux command line environment (regardless of your current computational environment) to perform all analysis in command shells. This works for all the tutorials on this page. Additionally, some tutorials support command "statgen-setup launch" which will start a JupyterLab server to perform the analysis. | ||
==Alohomora== | ==Alohomora== | ||
| − | * [http://statgen.us/files/tutorials | + | * [http://statgen.us/files/tutorials/Alohomora_Exercise_revised.pdf Exercise <nowiki>[PDF]</nowiki>] |
* [https://statgen.research.bcm.edu/files/2016/10/data/alohomora.zip Data Set] | * [https://statgen.research.bcm.edu/files/2016/10/data/alohomora.zip Data Set] | ||
* [http://gmc.mdc-berlin.de/alohomora/ Software Link] | * [http://gmc.mdc-berlin.de/alohomora/ Software Link] | ||
| − | == | + | ==Annovar complex traits== |
| − | * [https://github.com/statgenetics/statgen-courses/blob/master/handout/FunctionalAnnotation. | + | * [https://github.com/statgenetics/statgen-courses/blob/master/handout/FunctionalAnnotation.pdf Functional Annotation Exercise <nowiki>[PDF]</nowiki>] |
* [[Commands in Annotation Exercise|Exercise Commands]] | * [[Commands in Annotation Exercise|Exercise Commands]] | ||
| Line 25: | Line 37: | ||
</pre> | </pre> | ||
| − | + | ==Annovar Mendelian traits== | |
| − | + | ||
| − | ==Annovar | + | |
* [http://statgen.us/files/tutorials/FunctionalAnnotation_Annovar_final.pdf Exercise <nowiki>[PDF]</nowiki>] | * [http://statgen.us/files/tutorials/FunctionalAnnotation_Annovar_final.pdf Exercise <nowiki>[PDF]</nowiki>] | ||
| − | * [https://statgen. | + | * [https://statgen.us/files/2017/09/commands/annovar-functional_annotation.txt Commands Part I - Functional Annotation] |
| − | * [https://statgen. | + | * [https://statgen.us/files/2017/09/commands/annovar-variant_filtering.txt Commands Part II - Variant Filtering] |
==Cochran Armitage Trend Test== | ==Cochran Armitage Trend Test== | ||
| Line 44: | Line 54: | ||
<pre>statgen-setup login --tutorial epistasis | <pre>statgen-setup login --tutorial epistasis | ||
</pre> | </pre> | ||
| − | |||
| − | |||
| − | |||
| Line 57: | Line 64: | ||
</pre> | </pre> | ||
| − | + | ||
| + | ==Fine-mapping (SuSiE method)== | ||
| + | * [https://github.com/statgenetics/statgen-courses/blob/master/handout/finemapping.docx susieR Exercise <nowiki>[DOCX]</nowiki>] | ||
| + | * [https://github.com/statgenetics/statgen-courses/blob/master/handout/finemapping_answers.docx susieR Exercise Answers <nowiki>[DOCX]</nowiki>] | ||
| + | * [https://github.com/statgenetics/statgen-courses/blob/master/notebooks/finemapping.ipynb susieR Exercise <nowiki>[Ipython notebook]</nowiki>] | ||
| + | * [https://github.com/statgenetics/statgen-courses/blob/master/notebooks/finemapping_answers.ipynb susieR Exercise Answers <nowiki>[Ipython notebook]</nowiki>] | ||
| + | |||
| + | |||
| + | To run the exercise from docker image provided, | ||
| + | |||
| + | <pre>statgen-setup launch --tutorial finemap | ||
| + | </pre> | ||
==GCTA== | ==GCTA== | ||
| Line 67: | Line 85: | ||
</pre> | </pre> | ||
| − | |||
==Gemini== | ==Gemini== | ||
| Line 76: | Line 93: | ||
<pre>statgen-setup login --tutorial gemini | <pre>statgen-setup login --tutorial gemini | ||
</pre> | </pre> | ||
| − | |||
| − | |||
| − | |||
| Line 92: | Line 106: | ||
==GWAS: Data Quality Control== | ==GWAS: Data Quality Control== | ||
| − | * [https://github.com/statgenetics/statgen-courses/blob/master/handout/PLINK_data_QC. | + | * [https://github.com/statgenetics/statgen-courses/blob/master/handout/PLINK_data_QC.pdf Exercise <nowiki>[PDF]</nowiki>] |
| + | * [https://github.com/statgenetics/statgen-courses/blob/master/notebooks/PLINK_Data_QC.ipynb Exercise <nowiki>[IPython Notebook]</nowiki>] | ||
* [[GWAS Data QC Exercise|Exercise Commands]] | * [[GWAS Data QC Exercise|Exercise Commands]] | ||
| − | To run the exercise from docker image provided, | + | To run the exercise via JupyterLab from docker image provided, |
| − | <pre>statgen-setup | + | <pre>statgen-setup launch --tutorial plink |
</pre> | </pre> | ||
| − | + | Alternatively, you can opt to run the exercise via command terminal from docker image provided, | |
| − | + | ||
| + | <pre>statgen-setup login --tutorial plink | ||
| + | </pre> | ||
==GWAS: Association Analysis Controlling for Population Substructure== | ==GWAS: Association Analysis Controlling for Population Substructure== | ||
| − | * [https://github.com/statgenetics/statgen-courses/blob/master/handout/PLINK_Substructure. | + | * [https://github.com/statgenetics/statgen-courses/blob/master/handout/PLINK_Substructure.pdf Exercise <nowiki>[PDF]</nowiki>] |
| + | * [https://github.com/statgenetics/statgen-courses/blob/master/notebooks/PLINK_Substructure.ipynb Exercise <nowiki>[IPython Notebook]</nowiki>] | ||
* [[GWAS_Controlling_for_Population_Substructure|Exercise Commands]] | * [[GWAS_Controlling_for_Population_Substructure|Exercise Commands]] | ||
| − | To run the exercise from docker image provided, | + | To run the exercise via JupyterLab from docker image provided, |
| − | <pre>statgen-setup | + | <pre>statgen-setup launch --tutorial plink |
</pre> | </pre> | ||
| − | + | Alternatively, you can opt to run the exercise via command terminal from docker image provided, | |
| − | + | ||
| + | <pre>statgen-setup login --tutorial plink | ||
| + | </pre> | ||
==Homozygosity Mapper== | ==Homozygosity Mapper== | ||
| Line 126: | Line 144: | ||
* [http://statgen.us/files/igv_exercise.zip Exercise files (VCF and BAM)] | * [http://statgen.us/files/igv_exercise.zip Exercise files (VCF and BAM)] | ||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
==Linkage/FastLinkage== | ==Linkage/FastLinkage== | ||
| Line 142: | Line 155: | ||
</pre> | </pre> | ||
| − | + | ==LD clumping== | |
| + | * [https://github.com/cumc/bioworkflows/blob/master/GWAS/LD_Clumping.ipynb LD Clumping Exercise <nowiki>[IPython Notebook]</nowiki>] | ||
| + | |||
| + | |||
| + | To run the exercise from docker image provided, | ||
| + | |||
| + | <pre>statgen-setup launch --tutorial clumping | ||
| + | </pre> | ||
| + | |||
| + | ==MR-JTI for TWAS== | ||
| + | * [https://github.com/statgenetics/statgen-courses/blob/master/notebooks/MR_JTI.ipynb MR-JTI Exercise <nowiki>[IPython Notebook]</nowiki>] | ||
| + | |||
| + | |||
| + | To run the exercise from docker image provided, | ||
| + | |||
| + | <pre>statgen-setup launch --tutorial twas | ||
| + | </pre> | ||
==Pleiotropy== | ==Pleiotropy== | ||
| − | * [https://github.com/statgenetics/statgen-courses/blob/master/handout/Pleiotropy. | + | * [https://github.com/statgenetics/statgen-courses/blob/master/handout/Pleiotropy.pdf Pleiotropy Exercise <nowiki>[PDF]</nowiki>] |
| − | * [https://github.com/statgenetics/statgen-courses/blob/master/handout/Pleiotropy_answers. | + | * [https://github.com/statgenetics/statgen-courses/blob/master/handout/Pleiotropy_answers.pdf Pleiotropy Answers to Questions <nowiki>[PDF]</nowiki>] |
To run the exercise from docker image provided, | To run the exercise from docker image provided, | ||
| Line 153: | Line 182: | ||
</pre> | </pre> | ||
| − | |||
==Polygenic risk prediction (NPS method)== | ==Polygenic risk prediction (NPS method)== | ||
| − | * [https://github.com/statgenetics/statgen-courses/blob/master/handout/NPS. | + | * [https://github.com/statgenetics/statgen-courses/blob/master/handout/NPS.pdf PRS NPS Exercise <nowiki>[DOCX]</nowiki>] |
To run the exercise from docker image provided, | To run the exercise from docker image provided, | ||
| Line 163: | Line 191: | ||
</pre> | </pre> | ||
| − | + | ||
| + | ==Polygenic risk prediction (LDpred2 method)== | ||
| + | * [https://github.com/statgenetics/statgen-courses/blob/master/handout/ldpred2_example.pdf PRS LDpred2 Exercise <nowiki>[PDF]</nowiki>] | ||
| + | * [https://github.com/cumc/bioworkflows/blob/master/ldpred/ldpred2_example.ipynb PRS LDpred2 Exercise <nowiki>[Ipython Notebook]</nowiki>] | ||
| + | |||
| + | |||
| + | To run the exercise from docker image provided, | ||
| + | |||
| + | <pre>statgen-setup launch --tutorial ldpred2 | ||
| + | </pre> | ||
| + | |||
| + | Then follow prompts on the terminal output to open up the JupyterLab server in your web browser. If it is the first time you start this server, please open a command terminal inside JupyterLab, and type | ||
| + | |||
| + | <pre>get-data</pre> | ||
| + | |||
| + | to load the data-set to the JupyterLab workspace. | ||
| + | |||
<!-- | <!-- | ||
| Line 181: | Line 225: | ||
</pre> | </pre> | ||
| − | |||
==PSEQ== | ==PSEQ== | ||
| − | * [https://github.com/statgenetics/statgen-courses/blob/master/handout/PSEQ. | + | * [https://github.com/statgenetics/statgen-courses/blob/master/handout/PSEQ.pdf PSEQ Exercise <nowiki>[PDF]</nowiki>] |
| − | * [ | + | * [https://github.com/statgenetics/statgen-courses/blob/master/notebooks/PSEQ.ipynb PSEQ Exercise <nowiki>[Ipython Notebook]</nowiki>] |
To run the exercise from docker image provided, | To run the exercise from docker image provided, | ||
| + | |||
| + | <pre>statgen-setup launch --tutorial pseq | ||
| + | </pre> | ||
| + | |||
| + | Notice that since PSEQ exercise does not involve generating and visualizing plots, it is also fine to use a command terminal, instead of the JupyterLab server, to run this exercise and reproduce exactly what was described in the tutorial. To do so, | ||
<pre>statgen-setup login --tutorial pseq | <pre>statgen-setup login --tutorial pseq | ||
</pre> | </pre> | ||
| − | + | ||
| + | |||
| + | ==R and PLINK== | ||
| + | |||
| + | To run the exercise from docker image provided, | ||
| + | |||
| + | <pre>statgen-setup launch --tutorial plink-r-nothnagel | ||
| + | </pre> | ||
| + | |||
| + | |||
| + | ==REGENIE== | ||
| + | * [https://github.com/statgenetics/statgen-courses/blob/master/handout/regenie_example.pdf REGENIE Exercise <nowiki>[PDF]</nowiki>] | ||
| + | * [https://github.com/statgenetics/statgen-courses/blob/master/notebooks/regenie_example.ipynb REGENIE Exercise <nowiki>[Ipython Notebook]</nowiki>] | ||
| + | |||
| + | |||
| + | To run the exercise from docker image provided, | ||
| + | |||
| + | <pre>statgen-setup launch --tutorial regenie | ||
| + | </pre> | ||
| + | |||
| + | Then follow prompts on the terminal output to open up the JupyterLab server in your web browser. If it is the first time you start this server, please open a command terminal inside JupyterLab, and type | ||
| + | |||
| + | <pre>get-data</pre> | ||
| + | |||
| + | to load the data-set to the JupyterLab workspace. | ||
| + | |||
| + | |||
==Regression== | ==Regression== | ||
| Line 203: | Line 277: | ||
</pre> | </pre> | ||
| − | |||
==RV-TDT== | ==RV-TDT== | ||
| Line 242: | Line 315: | ||
To run the exercise from docker image provided, | To run the exercise from docker image provided, | ||
| − | <pre>statgen-setup login --tutorial slink | + | <pre>statgen-setup login --tutorial slink</pre> |
| − | </pre> | + | |
| − | + | ||
| − | + | ||
| − | + | ||
| − | + | ||
==SUPERLINK== | ==SUPERLINK== | ||
| Line 254: | Line 322: | ||
==Variant Association Tools== | ==Variant Association Tools== | ||
| − | * [https:// | + | * [https://github.com/statgenetics/statgen-courses/blob/master/handout/VAT.docx VAT Exercise <nowiki>[DOCX]</nowiki>] |
| − | * [ | + | * [https://github.com/statgenetics/statgen-courses/blob/master/notebooks/VAT.ipynb VAT Exercise <nowiki>[Ipython notebook]</nowiki>] |
To run the exercise from docker image provided, | To run the exercise from docker image provided, | ||
| − | <pre>statgen-setup | + | <pre>statgen-setup launch --tutorial vat |
</pre> | </pre> | ||
| − | + | Then follow the prompts on the terminal output to open up the JupyterLab server in your web browser. You should find the exercise notebook in the side panel, and you can click to open it. | |
Latest revision as of 16:06, 8 November 2022
Contents
- 1 Running Tutorials on Your Computer
- 2 Alohomora
- 3 Annovar complex traits
- 4 Annovar Mendelian traits
- 5 Cochran Armitage Trend Test
- 6 Epistasis (PLINK and CASSI)
- 7 FastLMM
- 8 Fine-mapping (SuSiE method)
- 9 GCTA
- 10 Gemini
- 11 Genehunter
- 12 GWAS: Data Quality Control
- 13 GWAS: Association Analysis Controlling for Population Substructure
- 14 Homozygosity Mapper
- 15 IGV
- 16 Linkage/FastLinkage
- 17 LD clumping
- 18 MR-JTI for TWAS
- 19 Pleiotropy
- 20 Polygenic risk prediction (NPS method)
- 21 Polygenic risk prediction (LDpred2 method)
- 22 Population Genetics
- 23 PSEQ
- 24 R and PLINK
- 25 REGENIE
- 26 Regression
- 27 RV-TDT
- 28 SEQLinkage
- 29 SEQSpark
- 30 SLINK
- 31 SUPERLINK
- 32 Variant Association Tools
Running Tutorials on Your Computer
Starting Fall 2019 we adopt docker to run our course material . We have created various docker repositories with source material freely available from github for users to readily setup and reproduce our tutorials on their own computers. These docker images can also be used as production tool to run relevant software on your computer (Mac, Linux or Windows) or even a high performance computing cluster (if properly configured) for your own data analysis.
General instructions
- Instructions to setup course tutorial environment on your computer
- Instructions to run course tutorial through JupyterLab
- Instructions to run course tutorial through command line terminal
Preparing Your Computer
Running Exercises
Tutorial specific instructions
We use a script "statgen-setup" to start the docker based environments for these tutorials. Please refer to the previous section for instructions on the installation of this script.
Material and instructions for specific exercise are listed in each section below (only those using statgen-setup command are relevant to our docker based tutorials). They provide links to materials and a minimal set of commands to use for launching and running an exercise. Command "statgen-setup login" will allow you to start and login to a Linux command line environment (regardless of your current computational environment) to perform all analysis in command shells. This works for all the tutorials on this page. Additionally, some tutorials support command "statgen-setup launch" which will start a JupyterLab server to perform the analysis.
Alohomora
Annovar complex traits
To run the exercise from docker image provided,
statgen-setup login --tutorial annovar
Annovar Mendelian traits
Cochran Armitage Trend Test
Epistasis (PLINK and CASSI)
To run the exercise from docker image provided,
statgen-setup login --tutorial epistasis
FastLMM
To run the exercise from docker image provided,
statgen-setup login --tutorial fastlmm-gcta
Fine-mapping (SuSiE method)
- susieR Exercise [DOCX]
- susieR Exercise Answers [DOCX]
- susieR Exercise [Ipython notebook]
- susieR Exercise Answers [Ipython notebook]
To run the exercise from docker image provided,
statgen-setup launch --tutorial finemap
GCTA
To run the exercise from docker image provided,
statgen-setup login --tutorial fastlmm-gcta
Gemini
To run the exercise from docker image provided,
statgen-setup login --tutorial gemini
Genehunter
To install from packages, follow the configuration steps above and run the following command.
sudo apt-get install genehunter-tutorial
The exercise's files will then be installed in the folder /home/shared/genehunter. You can run from there or copy the files into your user's home directory and proceed with the exercise.
GWAS: Data Quality Control
To run the exercise via JupyterLab from docker image provided,
statgen-setup launch --tutorial plink
Alternatively, you can opt to run the exercise via command terminal from docker image provided,
statgen-setup login --tutorial plink
GWAS: Association Analysis Controlling for Population Substructure
To run the exercise via JupyterLab from docker image provided,
statgen-setup launch --tutorial plink
Alternatively, you can opt to run the exercise via command terminal from docker image provided,
statgen-setup login --tutorial plink
Homozygosity Mapper
IGV
Linkage/FastLinkage
To run the exercise from docker image provided,
statgen-setup login --tutorial mlink
LD clumping
To run the exercise from docker image provided,
statgen-setup launch --tutorial clumping
MR-JTI for TWAS
To run the exercise from docker image provided,
statgen-setup launch --tutorial twas
Pleiotropy
To run the exercise from docker image provided,
statgen-setup login --tutorial pleiotropy
Polygenic risk prediction (NPS method)
To run the exercise from docker image provided,
statgen-setup login --tutorial nps
Polygenic risk prediction (LDpred2 method)
To run the exercise from docker image provided,
statgen-setup launch --tutorial ldpred2
Then follow prompts on the terminal output to open up the JupyterLab server in your web browser. If it is the first time you start this server, please open a command terminal inside JupyterLab, and type
get-data
to load the data-set to the JupyterLab workspace.
Population Genetics
To run the exercise from docker image provided,
statgen-setup login --tutorial popgen
PSEQ
To run the exercise from docker image provided,
statgen-setup launch --tutorial pseq
Notice that since PSEQ exercise does not involve generating and visualizing plots, it is also fine to use a command terminal, instead of the JupyterLab server, to run this exercise and reproduce exactly what was described in the tutorial. To do so,
statgen-setup login --tutorial pseq
R and PLINK
To run the exercise from docker image provided,
statgen-setup launch --tutorial plink-r-nothnagel
REGENIE
To run the exercise from docker image provided,
statgen-setup launch --tutorial regenie
Then follow prompts on the terminal output to open up the JupyterLab server in your web browser. If it is the first time you start this server, please open a command terminal inside JupyterLab, and type
get-data
to load the data-set to the JupyterLab workspace.
Regression
To run the exercise from docker image provided,
statgen-setup login --tutorial regression
RV-TDT
Installing Packages
To install from packages, follow the configuration steps above and run the following command.
sudo apt-get install rvtdt-tutorial
The exercise's files will then be installed in the folder /home/shared/rvtdt. You can run from there or copy the files into your user's home directory and proceed with the exercise.
SEQLinkage
To install from packages, follow the configuration steps above and run the following command.
sudo apt-get install seqlinkage-tutorial
The exercise's files will then be installed in the folder /home/shared/seqlinkage. You can run from there or copy the files into your user's home directory and proceed with the exercise.
SEQSpark
Installing Packages
To install from packages, follow the configuration steps above and run the following command.
sudo apt-get install seqspark-tutorial
The exercise's files will then be installed in the folder /home/shared/seqspark. You can run from there or copy the files into your user's home directory and proceed with the exercise.In order for the commands to work correctly, you don't need to reboot, but you should log out and log back in to make sure that the computer's environment is correctly configured.
SLINK
To run the exercise from docker image provided,
statgen-setup login --tutorial slink
SUPERLINK
Variant Association Tools
To run the exercise from docker image provided,
statgen-setup launch --tutorial vat
Then follow the prompts on the terminal output to open up the JupyterLab server in your web browser. You should find the exercise notebook in the side panel, and you can click to open it.
