django user tips

On this page, useful info is gathered for the usage of django.ugent.be.

Server Specs

  • RAM: 64 GiB (max: 64 GiB) DDR3-1600 RDIMM
  • CPU: Intel(R) Xeon(R) CPU E5-2630 0 @ 2.30GHz
  • HDD: 6 TB (HD3), 547GB (HD1), 275G (HD2)
  • OS: openSUSE LEAP 15.1

Info on recent server usage can be found on http://www.kennybogaert.eu/servers/.

There are 3 disk partitions:
1. HD1 contains the operating system and databases
2. HD2 only databases: such as nr: /media/HD2/
3. HD3 /home directories in /media/HD3/home/

LOGIN

First, connect to the UGent network via VPN connection or kabel.

*For Linux users:

Just login via your commandline:

ssh username@django.ugent.be

scripts can be run in the background using nohup

nohup command &
nohup command >/dev/null 2>&1 & # creates no nohup.out

Or when programs with a graphical user interface need to be loaded (Rstudio, CLC genomics, ..)

ssh -X username@django.ugent.be 

Programs can be run by executing the command:

eg.

For a good text editor:

scite

for ugene:

ugene -ui

or for clc genomics workbench:

clcgenomicswb10

*Windows users:

Windows users may install a free client as alternative. (https://mobaxterm.mobatek.net/)

FILETRANSFER

*For Linux users:

just via commandline

port 22 is open.

VIA SCP(commandline)Copying file to host:

scp /some/local/file.txt username@django.ugent.be:/some/remote/directory/
 Copying file from host to local directory:
scp username@django.ugent.be:/some/remote/file /some/local/directory/
Copying directory from host
scp -r user@django.ugent.be:/some/local/directory/   /some/local/directory/
2. VIA FILEZILLA(gui – sFTP)
host: django.ugent.be
user: yourusername
password: youraccountpassword
port: 22

*Windows users:

Windows users can install a free client: program WinSCP (https://winscp.net/eng/download.php), a free FTP and SCP client.

VNC

Virtual Network Computing (VNC) is a graphical desktop sharing system that to remotely control another computer. On Django a VNC session will start the KDE desktop as a graphic user interface. When you close your VNC viewer on your local desktop all programs will keep running on the server and your server session remains the same when you login the next day.

This is ideal for using program that run a graphic user interface such as UGENE,  CLC genomics workbench or Geneious.

UGENE and CLC work for all users.

Please contact if you wish to use Geneious.
or use the dedicated Geneious account:
account: geneious
password: geneious
vncpassword: geneious

Geneious seems to work faster using VNC than ssh -X (or mobaxterm).

Please note that if one user doesn’t close Geneious after using it, nobody can use it due to limitations of the licence! In this case, launching it from ssh -X (or mobaxterm) is a better habbit.

RealVNC:
install vncviewer from realvnc https://www.realvnc.com/download/viewer/)
1.connect via ssh to django.ugent.be
on linux:
ssh username@django.ugent.be
on windows: use putty (http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html)
2. Once logged in, start a vncserver session on the server using the following command:
vncserver :an_available_vnc_number

or 

vncserver
In the last case you will have to deduce your port number from the output:
(For some vncviewer versions you might want to adapt your screen resolution)
vncserver -geometry 1600x1200 :an_available_vnc_number
Please note: a vncserver session takes up a considerable amount of RAM memory. Please check you do not run multiple vncserver sessions with:
vncserver -list
3. connect using vncviewer
django.ugent.be:yourvncnumber

encryption: Let VNC choose
push “connect”
vncpassword invullen
Usual problem:
In case you are not able to launch a vnc session and you get an error message telling you there is session running, there is probably already a VNC session running on that port…
Try to kill it.
vncserver -kill :an_available_vnc_number
If it tells you, you don’t have the right to kill it or it doesn’t find the .pid file, it probably is launched by another user.
You can check all vncsessions running (and their ports) for example with the command:
ps -ef | grep "vnc"
Sometimes the graphics of kde may show problems (difficult to click icons, disappearing windows, commands in konsole not visible,… ). In this case you’ll need to kill your vncserver session and start a new one.
If this doesn’t help, you ‘ll have to contact bogaert.kenny@gmail.com. Restarting from runlevel 3 might do the trick (as root).
init 3

init 5 && exit

KDE

You can personalise you KDE desktop, for example:
-Please take notice of the handy widgets to track CPU, RAM and hard disk space. Right-click on the task bar > unlock widgets > add widgets > browse for handy widgets
-To get rid of the annoying apper message “Authentication is required to refresh the system resources”: Click on the application launcher > System settings > System administration > Startup and shutdown > Service Manager > disable “Apper monitor” and press ok. After you have relaunched your server session, this new setting will take effect.

Hard drive usage:

IMPORTANT: Each months, you should receive an automatic email listing all files that are older than 3 months and got categorized as expired. In case there is no other option to clear space and ensure a working server, a script will be run that permanently deletes these expired files… Therefore don’t rely on the server for data storage!

Files in ~/bin can be older of course. The ~/bin can ONLY be used for your binaries though: your personal scripts and programs. Don’t use it for real data storage.

RAM & CPU usage

1. If you overload any server it will crash at some point.  CLC bio in particular and multithreaded processes are very good at doing this.
Please keep track of top or htop if you plan to launch a process on multiple nodes. Or install a CPU & RAM monitoring widget in your KDE environment.
It is a waste to leave nodes open if they are unused. On the other hand use it with some common sense and moderation.
2. Close your VNC server if you do not use it for a while (Not just the screen but the server session via vncserver -kill). That can save some RAM memory.

NCBI – nr database

NCBI’s nr database is accessible in /home/nr/
blastx -query seq.fasta -db /home/nr/nr  -out output.xml -outfmt 5

List of installed programs

GUI Workbenches:

  • UGENE (v 1.31)
  • CLC Genomics Workbench (v10)
  • Geneious (v8.1.7)

Pipelines/Scripts:

Note: This inventory is incomplete

  • Abyss
  • BLAST+
  • bamtools
  • blastall (legacy blast)
  • CD-HIT
  • fastq-dump (v2.9.6)
  • fastq-join
  • flash (1.2.11)
  • FASTX toolkit (0.0.13)
  • Qiime1
  • Qiime2 (v2-2018.6, v2-2019.4)
  • Lotus  [version 1.38]
  • nohup
  • mira v4
  • Mothur (1.32.1)
  • PEAR (v0.9.5)
  • picrust2
  • prefetch (v2.9.3)
  • Rstudio (v 1.3.1056)
  • Samtools (v1.3)
  • SLR
  • Stacks (v2.4)
  • Trinity
  • usearch (UParse)
    • usearch [version 5.2.236] as usearch
    • usearch [version 6.1.544] as usearch61
    • usearch [version 8.0.1623_i86] as usearch8
    • usearch [version 8.1.1861_i86] as usearch81

Extra notes for specific program and pipelines

CLC workbench Updates

Desktop shortcuts for installed updated versions of CLC are made available in /opt/all_users_updates/ . You can launch them here (as sh script) or copy them here to your ~/Desktop

QIIME1 (1.9.1)

add to $HOME/.bashrc the following line

export PATH="/opt/anaconda/bin:$PATH"

Anytime you want to use QIIME after installation with Miniconda, you’ll need to reactivate your qiime1 environment using this command:

source activate qiime1

To leave the conda environment:

conda deactivate

QIIME2 (v2-2018.6; v2-2019.4)

add to $HOME/.bashrc the following line

export PATH="/opt/anaconda/bin:$PATH"

Anytime you want to use QIIME after installation with Miniconda, you’ll need to reactivate your qiime2 environment using this command:

source activate qiime2-2018.6

or

source activate qiime2-2019.4

To leave the conda environment:

conda deactivate

Picrust 2

add to $HOME/.bashrc the following line

export PATH="/opt/anaconda/bin:$PATH"

Anytime you want to use QIIME after installation with Miniconda, you’ll need to reactivate your picrust2 environment using this command:

source activate picrust2

To leave the conda environment:

conda deactivate

Lotus pipeline:

accessible in path /mnt/lotus_pipeline/ eg. :

perl /mnt/lotus_pipeline/lotus.pl -h