– django user tips

On this page, useful info is gathered for the usage of django.ugent.be.

Server Specs

  • RAM: 64 GiB (max: 64 GiB) DDR3-1600 RDIMM
  • CPU: Intel(R) Xeon(R) CPU E5-2630 0 @ 2.30GHz
  • HDD: 6 TB (HD3), 547GB (HD1), 275G (HD2)
  • OS: openSUSE LEAP 15.2

Info on recent server usage can be found on http://www.kennybogaert.eu/servers/.

There are 3 disk partitions:
1. HD1 contains the operating system and databases
2. HD2 only databases: such as nr: /media/HD2/
3. HD3 /home directories in /media/HD3/home/

LOGIN

First, connect to the UGent network via VPN connection or ftp cable.

*For Linux users:

Just login via your commandline:

ssh username@django.ugent.be

scripts can be run in the background using nohup

nohup command &
nohup command >/dev/null 2>&1 & # creates no nohup.out

Or when programs with a graphical user interface need to be loaded (Rstudio, CLC genomics, ..)

ssh -X username@django.ugent.be 

Programs can be run by executing the command:

eg.

For a good text editor:

scite

for ugene:

ugene -ui

or for clc genomics workbench:

clcgenomicswb10

*Windows users:

Windows users may install a free client as alternative. (https://mobaxterm.mobatek.net/)

FILETRANSFER

*For Linux users:

just via commandline

port 22 is open.

VIA SCP(commandline)Copying file to host:

scp /some/local/file.txt username@django.ugent.be:/some/remote/directory/
 Copying file from host to local directory:
scp username@django.ugent.be:/some/remote/file /some/local/directory/
Copying directory from host
scp -r user@django.ugent.be:/some/local/directory/   /some/local/directory/
2. VIA FILEZILLA(gui – sFTP)
host: django.ugent.be
user: yourusername
password: youraccountpassword
port: 22

*Windows users:

Windows users can install a free client: program WinSCP (https://winscp.net/eng/download.php), a free FTP and SCP client.

VNC

Virtual Network Computing (VNC) is a graphical desktop sharing system that to remotely control another computer. On Django a VNC session will start the xfce4 desktop as a graphic user interface. When you close your VNC viewer on your local desktop all programs will keep running on the server and your server session remains the same when you login the next day.

This is ideal for using program that run a graphic user interface such as UGENE, CLC genomics workbench or Geneious.

UGENE and CLC work for all users.

Please contact if you wish to use Geneious.
or use the dedicated Geneious account:
account: geneious
password: geneious
vncpassword: geneious

Geneious seems to work faster using VNC than ssh -X (or mobaxterm).

RealVNC:
install vncviewer from realvnc https://www.realvnc.com/download/viewer/)
1.connect via ssh to django.ugent.be
on linux:
ssh username@django.ugent.be
on windows: use putty (http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html)
2. Once logged in, start a vncserver session on the server using the following command:
vncserver :an_available_vnc_number

or 

vncserver
In the last case you will have to deduce your port number from the output:
(For some vncviewer versions you might want to adapt your screen resolution)
vncserver -geometry 1600x1200 :an_available_vnc_number
Please note: a vncserver session takes up a considerable amount of RAM memory. Please check you do not run multiple vncserver sessions with:
vncserver -list
3. connect using vncviewer
django.ugent.be:yourvncnumber

encryption: Let VNC choose
push “connect”
vncpassword invullen
Usual problem:
In case you are not able to launch a vnc session and you get an error message telling you there is session running, there is probably already a VNC session running on that port…
Try to kill it.
vncserver -kill :an_available_vnc_number
If it tells you, you don’t have the right to kill it or it doesn’t find the .pid file, it probably is launched by another user.
You can check all vncsessions running (and their ports) for example with the command:
ps -ef | grep "vnc"

Hard drive usage:

IMPORTANT: The server only can provide temporary working space! Please use a data server (eg. Ornette, UGENT shares …), cloud servers (eg. UGent One Drive,..) or external disks.

Each months, you should receive an automatic email listing all files that are older than 3 months and got categorized as expired. In case there is no other option to clear space and ensure a working server, a script will be run that permanently deletes these expired files… Therefore don’t rely on the server for data storage!

Files in ~/bin can be older of course. The ~/bin can ONLY be used for your binaries though: your personal scripts and programs. Don’t use it for real data storage.
If you want direct access via your vncsession from django to the UGENT share(s) of your research group in order to directly transfer files for storage:
1. Open File manager and browse to:
smb://<yourUGENTusername>@files.ugent.be/<yourUGENTusername>/
2. Fill in when requested for workdomain and password:
– workdomain: UGENT
– Password: your ugent password
Please note: you don’t need to fill the password and username of your django account, but that of your ugent account
You will get access to your ugent shares in your vncsession:
3. Click on the folder shares

RAM & CPU usage

1. If you overload any server it will crash at some point.  CLC bio in particular and multithreaded processes are very good at doing this.
Please keep track of top or htop if you plan to launch a process on multiple nodes. Or install a CPU & RAM monitoring widget in your KDE environment.
It is a waste to leave nodes open if they are unused. On the other hand use it with some common sense and moderation.
2. Close your VNC server if you do not use it for a while (Not just the screen but the server session via vncserver -kill). That can save some RAM memory.

NCBI – nr database

NCBI’s nr database is accessible in /home/nr/
blastx -query seq.fasta -db /home/nr/nr  -out output.xml -outfmt 5

List of installed programs

GUI Workbenches:

  • UGENE (v 1.31)
  • CLC Genomics Workbench (v10)
  • Geneious (v8.1.7)

Pipelines/Scripts:

Note: This inventory is incomplete

  • Abyss
  • BLAST+
  • bamtools
  • blastall (legacy blast)
  • CD-HIT
  • fastq-dump (v2.9.6)
  • fastq-join
  • flash (1.2.11)
  • FASTX toolkit (0.0.13)
  • Qiime1
  • Qiime2 (v2-2018.6, v2-2019.4)
  • Lotus  [version 1.38]
  • nohup
  • mira v4
  • Mothur (1.32.1)
  • PEAR (v0.9.5)
  • picrust2
  • prefetch (v2.9.3)
  • R (3.5.0 /usr/bin/R and 4.0.2 /opt/R-4.0.2/Bin/R)
  • Rstudio (v 1.3.1056)
  • Samtools (v1.3)
  • SLR
  • Stacks (v2.4)
  • Trinity
  • usearch (UParse)
    • usearch [version 5.2.236] as usearch
    • usearch [version 6.1.544] as usearch61
    • usearch [version 8.0.1623_i86] as usearch8
    • usearch [version 8.1.1861_i86] as usearch81

Extra notes for specific program and pipelines

CLC workbench Updates

Desktop shortcuts for installed updated versions of CLC are made available in /opt/all_users_updates/ . You can launch them here (as sh script) or copy them here to your ~/Desktop

QIIME1 (1.9.1)

add to $HOME/.bashrc the following line

export PATH="/opt/anaconda/bin:$PATH"

Anytime you want to use QIIME after installation with Miniconda, you’ll need to reactivate your qiime1 environment using this command:

source activate qiime1

To leave the conda environment:

conda deactivate

QIIME2 (v2-2018.6; v2-2019.4)

add to $HOME/.bashrc the following line

export PATH="/opt/anaconda/bin:$PATH"

Anytime you want to use QIIME after installation with Miniconda, you’ll need to reactivate your qiime2 environment using this command:

source activate qiime2-2018.6

or

source activate qiime2-2019.4

To leave the conda environment:

conda deactivate

Picrust 2

add to $HOME/.bashrc the following line

export PATH="/opt/anaconda/bin:$PATH"

Anytime you want to use QIIME after installation with Miniconda, you’ll need to reactivate your picrust2 environment using this command:

source activate picrust2

To leave the conda environment:

conda deactivate

Rstudio

If you want to change the R version to 4.0.2 or 3.6.1 (for OpenSUSE 15.2 by default 3.5)

export RSTUDIO_WHICH_R=/opt/R-4.0.2/bin/R
rstudio

or

export RSTUDIO_WHICH_R=/opt/R-3.6.1/bin/R
rstudio

or add the first line to your $HOME/.bashrc file or profile

Lotus pipeline:

accessible in path /mnt/lotus_pipeline/ eg. :

perl /mnt/lotus_pipeline/lotus.pl -h