wiki:BIOS_StorageInTheCloud

Version 1 (modified by jamverlouw, 9 months ago) (diff)

--

Storage in the cloud

We have limited space in the cloud available for analysis and storage.

Home folders

The /home/ folder on each VM has very limited disk space and should not hold any data or output.

We have a 2GB soft disk quota limit and 3GB hard disk quota limit per user.

The root system is only backed up irregularly, do not rely on this and please backup your own files from your home folder. More on backup of the virdir can be read at the bottom of this page.

VirDir

We have 1.5 Tb of shared space available. This disk is mounted as the /virdir/ folder and can be accessed from every VMs. There are no restrictions on who can read the data.

Scratch

/virdir/scratch/

Please make a folder with your username in /virdir/scratch/ for your data and analysis. For your convenience some data available on the SRM is downloaded to this folder already.

WWW export

/virdir/Scratch/wwwexport/

This directory is exposed over HTTP to the internet at the root of the virtual machine. This means the contents of this directory can be accessed over the internet by prefixing their filenames with http://bios-vm.bbmrirp3-lumc.vm.surfsara.nl/.

The primary purpose of this is for storing custom tracks for the UCSC Genome Browser.

Please note that no privacy sensitive data should be stored here, as it will be world-readable.

Backup

/virdir/Backup/

Data in the backup folder will be backed up nightly. Read below.

Storage policy

Please read and understand the next text from Sara.nl before using this space.

Images on the HPC Cloud system are not backed up. The HPC Cloud is a form of Infrastructure as a Service (IaaS), which means that the user must perform administrative tasks on her virtual machines or cluster. This includes the responsibility of taking care of backups. VirDir can be used to help with backups. VirDirs allow users to save their data outside their image and provide the option of a backup by SURFsara (by using the backup directory). Backups are made incrementally on a nightly basis. We can restore files and/or directories when you accidentally remove them (provided they already existed during the last successful backup). The retention period for recovering deleted files is 30 days. Every day, the content of your /Backup directory will be backed up by SARA. The /Scratch directory will never be backed up. You are the best person to judge what data should or should not be backed up and we ask you to use your best judgement.