Important locations¶
Below is an overview of locations Ponyland users visit often. If you
want to know more about the various disks these locations are on (for
example, how much space is left there), use the command df -h.
TL;DR where do I store my data?¶
| Location | Accessibility | Speed | Amount of space |
|---|---|---|---|
/home/user |
easily accessible from every pony | slow reading and writing | limited space |
/vol/tensusers(2-5)/user |
accessible from every pony | slow reading and writing | several TBs free |
/scratch/user |
different for every pony | fast reading and writing | space free differs per pony |
Where do I put my large machine learning models and data?¶
If you'd lke to store data or models that are >500 GB, please send an email to admin@cls.ru.nl with the following info:
- How much storage space do you need?
- How long do you need it?
Full overview¶
| Path | Disk name | Description |
|---|---|---|
~ |
lv-00 and lv-01 | Your home directory. Home directories are small, and should only be used for scripts and other small things. Not for custom installations of software. For this, please use /vol/tensusers. |
/scratch |
Individual machine | These disks are local, so their content is not shared with other machines. This disk can be used to temporarily write files that only exist during your experiment, or if you need fast IO for your experiments, you can store your data locally. Space is limited so please be considerate. |
/vol/bigdata/corpora |
bigdata-srv | Corpora |
/vol/bigdata/corpora2 |
bigdata2-srv | More corpora |
/vol/bigdata/datasets |
bigdata-srv | Datasets |
/vol/bigdata/datasets2 |
bigdata2-srv | More datasets |
/vol/bigdata/users |
bigdata-srv | Deprecated Personal data went here. We now have more space available on /vol/tensusers, please move your data there. |
/vol/tensusers |
tensusers-srv | Personal data goes here. |
/vol/tensusers2 |
tensusers2-srv | Extension for the original tensusers disk. |
/vol/tensusers3 |
tensusers3-srv | Extension for the original tensusers disk. |
/vol/tensusers4 |
tensusers4-srv | Extension for the original tensusers disk. |
/vol/tensusers5 |
tensusers5-srv | Extension for the original tensusers disk. |
/vol/tensusers6 |
tensusers6 | Extension for the original tensusers disk. |
/vol/tensusers7 |
tensusers7 | Extension for the original tensusers disk. |
/vol/tensusers8 |
tensusers8 | Extension for the original tensusers disk. |
/vol/mlusers |
mlusers | Big data/models for machine learning goes here. |
/www/lst/live/htdocs/staff |
lightning:/var/www2 | Personal Homepage |
And this how it's configured under the hood:
| Disk | Storage capacity | Physically linked to Pony |
|---|---|---|
| bigdata-srv | 11T | derpy |
| bigdata2-srv | 23T | derpy |
| customopt | 197G | ? |
| lv-00 | 300G | derpy |
| lv-01 | 296G | derpy |
| mlparchief-srv | 10T | daisy |
| tensusers-srv | 46T | daringdo |
| tensusers2-srv | 41T | derpy |
| tensusers3-srv | 10T | daisy |
| tensusers4-srv | 10T | daisy |
| tensusers5-srv | 10T | daisy |
| tensusers6 | 10T | dinky |
| tensusers7 | 10T | dinky |
| tensusers8 | 10T | dinky |
| mlusers | 100T | dinky |
| signlang-srv | 20T | daringdo |
| signlang2-srv | 22T | daringdo |
| signlang3-srv | 30T | daringdo |
| lightning:/var/www | 197G | lightning |