Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision		Previous revision
Next revisionBoth sides next revision
how_to [2023/05/17 19:48] – [Download files when you cannot login using a GUI?] adminhow_to [2024/01/24 17:12] – [Measure memory used by GPUa?] habil
Line 4: Line 4:
  
 ---- ----
 +
 +==== Measure memory used by GPUa? ====
 +
 +Use ''torch.cuda.[[https://pytorch.org/blog/understanding-gpu-memory-1/?hss_channel=tw-776585502606721024|memory]]'' , e.g., to discover the effect of clearing gradients at the end of each [[https://towardsdatascience.com/epoch-vs-iterations-vs-batch-size-4dfb9c7ce9c9|iteration]]. With ignite, this can be done using ''optimizer.[[https://stackoverflow.com/questions/48001598/why-do-we-need-to-call-zero-grad-in-pytorch|zero_grad]](set_to_none=True)''.'' ''
 +
 +----
 +
 +
 +==== Analyze multiomicds data? ====
 +
 +[[http://biggsinstitute.org/neurepiomics-2023/#1686596557113-fac33f90-1709|Neurepiomics]] workshop in 2023:
 +
 +  - Shiva's presentation on omics[[https://docs.google.com/presentation/d/1W-pJfPy5MkQscPn5iOSJAkOzuyC2-eirBeHHeKHuYyw/edit#slide=id.p|databases]]
 +  - Mohsen's [[https://docs.google.com/presentation/d/14z2jAkYfmyH6dIH0Rr59ywZk--Kk9ocHzWEDiyd_P4k/edit#slide=id.p|presentation]] for Neurepiomics workshop, 2023-10-09.
 +  - Habil's [[https://docs.google.com/presentation/d/1mGld52Y8TkMXArc5EhcD2XY_0rXvb79BFEgA0YaZwms/edit#slide=id.g63f2c216dd_0_0|slides]] on multiomics data integration.
 +
 +----
 +
 +
 +==== What is the best Linux distribution for biologists? ====
 +
 +[[https://archlinux.org/|ArchLinux]] and [[https://wiki.archlinux.org/title/Arch-based_distributions|its derivatives]], like [[https://manjaro.org/|Manjaro]], are the top Linux distributions for biologists. The [[https://github.com/BioArchLinux/Packages|BioArchLinux community]] maintains an excellent repository with over 4,000 bioinformatics packages and their dependencies. Regular automated updates ensure the latest versions are available by fetching source code from trusted repositories and compiling releases optimized for the target architecture. Its philosophy is described [[https://f1000research.com/posters/11-809|here]].
 +
 +----
 +
  
 ==== Download files to where you cannot login or click using a GUI? ==== ==== Download files to where you cannot login or click using a GUI? ====
Line 14: Line 39:
 ==== Supercharge your YouTube experience and minimize ads? ==== ==== Supercharge your YouTube experience and minimize ads? ====
  
-"[[https://chrome.google.com/webstore/detail/youtube-enhancer/ejcniippeghnejiodjkkmndlelbagmah|Enhancer]] for YouTube" is a popular browser extension that enhances the YouTube viewing experience by providing additional features and customization options. While it doesn't explicitly focus on blocking ads, it offers some options to minimize or hide them.+"[[https://chrome.google.com/webstore/detail/youtube-enhancer/ejcniippeghnejiodjkkmndlelbagmah|Enhancer]] for YouTube" is a popular browser extension that enhances the YouTube viewing experience by providing additional features and customization options. While it doesn't explicitly focus on blocking ads, it offers some options to minimize or hide them. Use [[https://chrome.google.com/webstore/detail/autoskip-for-youtube-ads/hmbnhhcgiecenbbkgdoaoafjpeaboine|Autoskip]] to automatically skip ads that pass the Enhancer filter.
  
 ---- ----
Line 39: Line 64:
 ==== Plot deep neural network architecture diagrams? ==== ==== Plot deep neural network architecture diagrams? ====
  
-There are several apps that would help in this manner which are listed based on their qualities:+There are [[https://datascience.stackexchange.com/questions/14899/how-to-draw-deep-learning-network-architecture-diagrams|several]] [[https://datascience.stackexchange.com/questions/12851/how-do-you-visualize-neural-network-architectures|apps]] that would help in this manner which are listed based on their descending qualities:
  
-  - [[https://github.com/HarisIqbal88/PlotNeuralNet|PlotNeuralNet]] (very simple, just generate fixed latex commands, no flexibility in the layers themselves)+  - [[https://github.com/HarisIqbal88/PlotNeuralNet|PlotNeuralNet]] (very simple, just generate fixed latex commands, no flexibility in the layers themselves, you need to rewrite models in this [[https://github.com/HarisIqbal88/PlotNeuralNet/blob/master/pyexamples/test_simple.py|language]]) {{https://user-images.githubusercontent.com/17570785/50308846-c2231880-049c-11e9-8763-3daa1024de78.png?direct&50x18}} 
 +  - [[https://analyticsindiamag.com/how-to-visualize-deep-learning-models-using-visualkeras/|Visualkeras]] (plots Keras models) {{https://149695847.v2.pressablecdn.com/wp-content/uploads/2021/11/image-37.png?direct&30x31}}
   - [[http://alexlenail.me/NN-SVG/index.html|NN-SVG]]   - [[http://alexlenail.me/NN-SVG/index.html|NN-SVG]]
-  - [[https://github.com/gwding/draw_convnet|draw_convnet]] +  - [[https://github.com/gwding/draw_convnet|draw_convnet]] {{https://raw.githubusercontent.com/gwding/draw_convnet/master/convnet_fig.png?direct&100x32}} 
-  - [[https://github.com/lutzroeder/netron|netron]] (does not consider ModuleList inside the model in PyTorch)+  - [[https://github.com/lutzroeder/netron|netron]] (does not consider ModuleList inside the model in PyTorch) {{https://github.com/lutzroeder/netron/raw/main/.github/screenshot.png?direct&50x34}}
   - [[https://viscom.net2vis.uni-ulm.de/|Net2Vis]] (automatically generates abstract visualizations for convolutional neural networks from Keras code)   - [[https://viscom.net2vis.uni-ulm.de/|Net2Vis]] (automatically generates abstract visualizations for convolutional neural networks from Keras code)
 +  - [[https://tikz.net/neural_networks/|LaTeX]] examples of neural networks.
  
 ---- ----
Line 154: Line 181:
  
 Use [[https://cran.r-project.org/web/packages/openxlsx/index.html|openxlsx]] package to read, write and edit xlsx files in R. Package's integration with C++ makes it faster and easier to use. Simplifies the creation of Excel .xlsx files by providing a high level interface to writing, styling and editing worksheets. Through the use of 'Rcpp', read/write times are comparable to the 'xlsx' and 'XLConnect' packages with the added benefit of removing the dependency on Java. Use [[https://cran.r-project.org/web/packages/openxlsx/index.html|openxlsx]] package to read, write and edit xlsx files in R. Package's integration with C++ makes it faster and easier to use. Simplifies the creation of Excel .xlsx files by providing a high level interface to writing, styling and editing worksheets. Through the use of 'Rcpp', read/write times are comparable to the 'xlsx' and 'XLConnect' packages with the added benefit of removing the dependency on Java.
- 
 <code> <code>
 +
 E.g. Writing four dataframes in four sheets of excel workbook can be done as follows: E.g. Writing four dataframes in four sheets of excel workbook can be done as follows:
 library(openxlsx) library(openxlsx)
Line 167: Line 194:
 writeData(wb=w1, sheet="New", mtcars[,1:3]) writeData(wb=w1, sheet="New", mtcars[,1:3])
 saveWorkbook(w1, file=xlsFile, overwrite=TRUE) saveWorkbook(w1, file=xlsFile, overwrite=TRUE)
 +
 +## Read a sheet in a data frame:
 +r1 <- read.xlsx(xlsxFile=xlsFile, sheet="Second")
 +
 </code> </code>
  
 ---- ----
 +
  
 ==== Set local mirror for Rscript ==== ==== Set local mirror for Rscript ====
Line 434: Line 466:
 ==== Use git via proxy or vpn? ==== ==== Use git via proxy or vpn? ====
  
-Use [[https://github.com/apenwarr/sshuttle|sshuttle]], e.g., sshuttle -r h_[[mailto:z14@nyx.cs.txstate.edu|mailto:z14@nyx.cs.txstate.edu]] 0.0.0.0/0 -vv \\+Use [[https://github.com/apenwarr/sshuttle|sshuttle]], e.g., sshuttle -r h_[[mailto:z14@nyx.cs.txstate.edu|mailto:z14@nyx.cs.txstate.edu]] 0.0.0.0/0 -vv\\
 The list of servers at Texas Sate University are listed [[https://cs.txstate.edu/resources/labs/accounts/linux/|here]]. The list of servers at Texas Sate University are listed [[https://cs.txstate.edu/resources/labs/accounts/linux/|here]].
  
 ---- ----
  
-**Enable autocomplete and the tab bar in Emacs?** \\ [[https://www.emacswiki.org/emacs/InstallingPackages#toc2|Install]] the auto-complete and tabbar packages from [[https://www.emacswiki.org/emacs/MELPA|MELPA]]. In OS X, you may need to edit your init file (usually it is .emacs but you can use C-H v user-init-file RET to [[https://stackoverflow.com/questions/189490/where-can-i-find-my-emacs-file-for-emacs-running-on-windows|check]]) file to remove auto-complete from the package-selected-packages list and instead add the following lines:+==== Enable autocomplete and the tab bar in Emacs? ====
  
 +\\
 +[[https://www.emacswiki.org/emacs/InstallingPackages#toc2|Install]] the auto-complete and tabbar packages from [[https://www.emacswiki.org/emacs/MELPA|MELPA]]. In OS X, you may need to edit your init file (usually it is .emacs but you can use C-H v user-init-file RET to [[https://stackoverflow.com/questions/189490/where-can-i-find-my-emacs-file-for-emacs-running-on-windows|check]]) file to remove auto-complete from the package-selected-packages list and instead add the following lines:
 <code> <code>
 +
 (require 'auto-complete) (require 'auto-complete)
 (global-auto-complete-mode t) (global-auto-complete-mode t)
 +
 </code> </code>
  
Line 455: Line 491:
 ---- ----
  
-**Resubmit an NIH grant?** \\ [[http://ctsi.ucla.edu/education/files/view/training/docs/study-sections-adams.pdf|Address]] ALL reviewers' comments. Write a strong [[http://www.regis.edu/~/media/Files/University/OAG/Learning-Resources/Create_Winning_Resubmission_Application_NIH_Report.ashx|introduction]] [{{:resubmittinggrantfelson10january2012.ppt|ppt}}  ]. Watch NIH [[https://public.csr.nih.gov/ForApplicants/InitialReviewResultsAndAppeals/csrwebinar|Webinars]] for Applicants.+**Resubmit an NIH grant?** \\ 
 +[[http://ctsi.ucla.edu/education/files/view/training/docs/study-sections-adams.pdf|Address]] ALL reviewers' comments. Write a strong [[http://www.regis.edu/~/media/Files/University/OAG/Learning-Resources/Create_Winning_Resubmission_Application_NIH_Report.ashx|introduction]] [{{:resubmittinggrantfelson10january2012.ppt|ppt}}  ]. Watch NIH [[https://public.csr.nih.gov/ForApplicants/InitialReviewResultsAndAppeals/csrwebinar|Webinars]] for Applicants.
  
 ---- ----
Line 464: Line 501:
 ---- ----
  
-**Compare the data resources on a gene across databases?** \\ [[https://xenabrowser.net/|Xena]] provides heatmaps for one gene using databases selected from a list of tens of databases from TCGA,1000 genomes, etc. Cancer Genome [[http://docs.cancergenomicscloud.org/|Cloud]] allows the user to integrate their own tools with theplatform and run it on, say, TCGA in the cloud. [[http://seek.princeton.edu/|Seeks]] orders thousands of datasets based on how concordant the expression of the input genes are.+**Compare the data resources on a gene across databases?** \\ 
 +[[https://xenabrowser.net/|Xena]] provides heatmaps for one gene using databases selected from a list of tens of databases from TCGA,1000 genomes, etc. Cancer Genome [[http://docs.cancergenomicscloud.org/|Cloud]] allows the user to integrate their own tools with theplatform and run it on, say, TCGA in the cloud. [[http://seek.princeton.edu/|Seeks]] orders thousands of datasets based on how concordant the expression of the input genes are.
  
 ---- ----
  
-**Save Powerpoint in high resolution on OS X?** \\ [[https://answers.microsoft.com/en-us/msoffice/forum/msoffice_powerpoint-mso_mac/how-do-i-save-powerpoint-slide-as-a-high/1baa460e-8ecb-4f44-a48f-425504d8d25a|Select]] all, open Preview > File > New from Clipboard. DPI is [[https://99designs.com/blog/tips/ppi-vs-dpi-whats-the-difference/|different]] from PPI.+**Save Powerpoint in high resolution on OS X?** \\ 
 +[[https://answers.microsoft.com/en-us/msoffice/forum/msoffice_powerpoint-mso_mac/how-do-i-save-powerpoint-slide-as-a-high/1baa460e-8ecb-4f44-a48f-425504d8d25a|Select]] all, open Preview > File > New from Clipboard. DPI is [[https://99designs.com/blog/tips/ppi-vs-dpi-whats-the-difference/|different]] from PPI.
  
 ---- ----
Line 488: Line 527:
  
 **Discover immunogenic antigens ([[http://www.nature.com.libproxy.uthscsa.edu/nrg/journal/v17/n8/full/nrg.2016.67.html?foxtrotcallback=true|neoantigens]]) suitable for immunotherapy?** \\ **Discover immunogenic antigens ([[http://www.nature.com.libproxy.uthscsa.edu/nrg/journal/v17/n8/full/nrg.2016.67.html?foxtrotcallback=true|neoantigens]]) suitable for immunotherapy?** \\
-Identify the somatic mutations in expressed genes, [[https://link.springer.com/article/10.1007%2Fs00262-012-1354-x|predict]] epitope //in silico//  using publicly available databases, and [[http://cancerres.aacrjournals.org/content/72/5/1081|immunize]] mice with long peptides encoding the mutated epitopes to determine immunogenicity.+Identify the somatic mutations in expressed genes, [[https://link.springer.com/article/10.1007%2Fs00262-012-1354-x|predict]] epitope //in silico// using publicly available databases, and [[http://cancerres.aacrjournals.org/content/72/5/1081|immunize]] mice with long peptides encoding the mutated epitopes to determine immunogenicity.
  
 ---- ----
  
-**Avoid misinterpretation of biological experiments?** \\ [[https://www.nature.com/nrc/journal/v17/n7/full/nrc.2017.32.html|Reasoning]] must be logical. Report enough details of the methods to reproduce the results. Assess the robustness of the findings with respect to minor perturbations to the experimental settings. To prove that drug A targets protein X, it is not sufficient to confirm that treatment with A leads to killing cells that have X. Maybe the cells are killed because of some other mechanism. Use "rescue experiments" as in the A=imatinib X=BCR–ABL case. Always, avoid [[https://elifesciences.org/articles/48175|these]] ten common statistical mistakes.+**Avoid misinterpretation of biological experiments?** \\ 
 +[[https://www.nature.com/nrc/journal/v17/n7/full/nrc.2017.32.html|Reasoning]] must be logical. Report enough details of the methods to reproduce the results. Assess the robustness of the findings with respect to minor perturbations to the experimental settings. To prove that drug A targets protein X, it is not sufficient to confirm that treatment with A leads to killing cells that have X. Maybe the cells are killed because of some other mechanism. Use "rescue experiments" as in the A=imatinib X=BCR–ABL case. Always, avoid [[https://elifesciences.org/articles/48175|these]] ten common statistical mistakes.
  
 ---- ----
  
-**Analyze single cell RNA-seq data****?** \\ [[https://hemberg-lab.github.io/scRNA.seq.course/introduction-to-single-cell-rna-seq.html|Map]] the reads [[https://www.nature.com/news/single-cell-sequencing-made-simple-1.22233|similar]] to bulk RNA-seq data, exclude [[https://hemberg-lab.github.io/scRNA.seq.course/construction-of-expression-matrix.html#identifying-confounding-factors|outlier]] cells and genes, [[https://hemberg-lab.github.io/scRNA.seq.course/biological-analysis.html#clustering-introduction|cluster]] the cells, and [[https://hemberg-lab.github.io/scRNA.seq.course/biological-analysis.html#projecting-scrna-seq-data|project]] the cells onto an annotated reference such as Human Cell Atlas. The broken links can be found in the [[https://www.singlecellcourse.org/index.html|Sanger's]] course.+**Analyze single cell RNA-seq data****?** \\ 
 +[[https://hemberg-lab.github.io/scRNA.seq.course/introduction-to-single-cell-rna-seq.html|Map]] the reads [[https://www.nature.com/news/single-cell-sequencing-made-simple-1.22233|similar]] to bulk RNA-seq data, exclude [[https://hemberg-lab.github.io/scRNA.seq.course/construction-of-expression-matrix.html#identifying-confounding-factors|outlier]] cells and genes, [[https://hemberg-lab.github.io/scRNA.seq.course/biological-analysis.html#clustering-introduction|cluster]] the cells, and [[https://hemberg-lab.github.io/scRNA.seq.course/biological-analysis.html#projecting-scrna-seq-data|project]] the cells onto an annotated reference such as Human Cell Atlas. The broken links can be found in the [[https://www.singlecellcourse.org/index.html|Sanger's]] course.
  
 ---- ----
Line 502: Line 543:
 **Make interactive 3D plots in R?** \\ **Make interactive 3D plots in R?** \\
 Use plot3D::[[http://www.sthda.com/english/wiki/impressive-package-for-3d-and-4d-graph-r-software-and-data-visualization#basic-scatter-plot|scatter3D]] to create the graph: Use plot3D::[[http://www.sthda.com/english/wiki/impressive-package-for-3d-and-4d-graph-r-software-and-data-visualization#basic-scatter-plot|scatter3D]] to create the graph:
- 
 <code> <code>
 +
 scatter3D(x=iris$Sepal.Length, y=iris$Petal.Length, z=iris$Sepal.Width, clab = c("Sepal", "Width (cm)")) scatter3D(x=iris$Sepal.Length, y=iris$Petal.Length, z=iris$Sepal.Width, clab = c("Sepal", "Width (cm)"))
 plotrgl() plotrgl()
 +
 </code> </code>
  
Line 512: Line 554:
 ---- ----
  
-**Improve the ranking of your institution better than MIT?!** \\ [[https://liorpachter.wordpress.com/2014/10/31/to-some-a-citation-is-worth-3-per-year/|Hire]] scholars who are frequently cited as adjunct professors.+**Improve the ranking of your institution better than MIT?!** \\ 
 +[[https://liorpachter.wordpress.com/2014/10/31/to-some-a-citation-is-worth-3-per-year/|Hire]] scholars who are frequently cited as adjunct professors.
  
 ---- ----
  
-**Rescue US biomedical research from its systemic flaws?** \\ [[http://www.pnas.org/content/111/16/5773|Hypercompetition]] has damaging effects. Reduce the number of PhD students and postdocs. Revise funding mechanisms to encourage "path-breaking ideas that, by definition," are [[https://www.nature.com/articles/426389a|risky]]. Support [[https://nexus.od.nih.gov/all/2018/05/04/the-issue-that-keeps-us-awake-at-night/|early]]-stage investigators.+**Rescue US biomedical research from its systemic flaws?** \\ 
 +[[http://www.pnas.org/content/111/16/5773|Hypercompetition]] has damaging effects. Reduce the number of PhD students and postdocs. Revise funding mechanisms to encourage "path-breaking ideas that, by definition," are [[https://www.nature.com/articles/426389a|risky]]. Support [[https://nexus.od.nih.gov/all/2018/05/04/the-issue-that-keeps-us-awake-at-night/|early]]-stage investigators.
  
 ---- ----
  
-**Test an R function that you have extracted from a script?** \\ +==== Test an R function that you have extracted from a script? ==== 
-Sometimes you have a working script but some part of it is useful also elsewhere. You extract that portion and make a function f(x,y) out of it. Now, in the script you call that function. You run your modified script and the output is exactly the same as before. Is this a sufficient test for the function you just wrote? No! What if you forget to include one of the variables as an input argument of the function? Then, that variable, which is still defined somewhere in the script before you call the function, is used in the function as a "global variable" without any explicit error. However, when the function is used in a different context, that global variable may not be defined, or worse, it may have an irrelevant value. To avoid this issue, test your function as follows:+ 
 +Sometimes you have a working script but some part of it is useful also elsewhere. You extract that portion and make a function f(x,y) out of it. Now, in the script you call that function. You run your modified script and the output is exactly the same as before. Is this a sufficient test for the function you just wrote? No! What if you forget to include one of the variables as an input argument of the function? Then, that variable, which is still defined somewhere in the script before you call the function, is used in the function as a "global variable" without any explicit error. However, when the function is used in a different context, that global variable may not be defined, or worse, it may have an irrelevant value. To avoid this issue, use codetools::[[https://rdrr.io/cran/codetools/man/findGlobals.html|findGlobals]](..., merge=FALSE) to identify and remove all global variables from your function. A cumbersome and less accurate way to identify global variables follows:
  
   - Put a browser in the script right before calling the function.   - Put a browser in the script right before calling the function.
Line 533: Line 578:
 ---- ----
  
-**Separate the reads corresponding to each individual cell from a single-cell RNA-Seq fastq file?** \\ [[http://www.arrayserver.com/wiki/index.php?title=Preprocess_of_SingleCell_RNA-Seq_Data|Use]] the barcodes and Unique Molecular Identifiers (UMIs).+ 
 +====   Separate the reads corresponding to each individual cell from a single-cell RNA-Seq fastq file?   ==== 
 + 
 + \\ [[http://www.arrayserver.com/wiki/index.php?title=Preprocess_of_SingleCell_RNA-Seq_Data|Use]] the barcodes and Unique Molecular Identifiers (UMIs).
  
 ---- ----
  
-**Copy from your prior clipboard history?** \\+====   Copy from your prior clipboard history?   ==== 
 Use a clipboard manager like [[https://clipy-app.com/|Clipy]] (developed based on ClipMenu) or JumpCut. Clipy allows you to save lists of favorite or frequent text snippets for later use. Use a clipboard manager like [[https://clipy-app.com/|Clipy]] (developed based on ClipMenu) or JumpCut. Clipy allows you to save lists of favorite or frequent text snippets for later use.
  
 ---- ----
  
-**Review a paper?** \\+====   Review a paper?   ==== 
 Read the journal guidelines, and the Nature's quick and concise [[https://www.springer.com/gp/authors-editors/authorandreviewertutorials/howtopeerreview?utm_source=hybris&utm_medium=email&utm_content=internal&utm_campaign=AEXS_1_PF_ReviewerThankYou_tutorial&sap-outbound-id=37A4EBB275E142189BFB59083911E79615029EBE|tutorial]]. The PLOS Open Reviewer [[https://genweb.plos.org/RR/PLOSOpenReviewerGatewayInfosheet.pdf|Gateway]] can be helpful if you are interested in becoming a reviewer for PLOS journals. Read the journal guidelines, and the Nature's quick and concise [[https://www.springer.com/gp/authors-editors/authorandreviewertutorials/howtopeerreview?utm_source=hybris&utm_medium=email&utm_content=internal&utm_campaign=AEXS_1_PF_ReviewerThankYou_tutorial&sap-outbound-id=37A4EBB275E142189BFB59083911E79615029EBE|tutorial]]. The PLOS Open Reviewer [[https://genweb.plos.org/RR/PLOSOpenReviewerGatewayInfosheet.pdf|Gateway]] can be helpful if you are interested in becoming a reviewer for PLOS journals.
  
 ---- ----
  
-**Adjust the acceleration and speed of a Logitec mouse?** \\  \\ +====   Adjust the acceleration and speed of a Logitec mouse?   ==== 
-Install and [[https://apple.stackexchange.com/questions/253111/how-to-disable-scroll-acceleration-in-macos-sierra|use]] the Logitech Control Center.+ 
 + \\ Install and [[https://apple.stackexchange.com/questions/253111/how-to-disable-scroll-acceleration-in-macos-sierra|use]] the Logitech Control Center.
  
 ---- ----
 +
  
 ==== Encrypt a folder? ==== ==== Encrypt a folder? ====