Need Help?

Live Distribution

Welcome to the documentation for using the Live Distribution feature for distributing data files securely through the EGA platform. This guide will walk you through the process of downloading encrypted files and decrypting them using Crypt4GH. Please follow the steps below to ensure a smooth experience.

Before Downloading

  1. Create an EGA user.
  2. Make sure that you have the permissions to download the dataset of interest. In case you don’t have access, request access to the dataset.
  3. Add your Crypt4GH-compatible public key to your EGA account. As soon as you save your public key, you will be able to connect to this EGA outbox using the SFTP protocol.

Download

Graphical User Interface (GUI)

You can use any GUI that supports SFTP connections, such as FileZilla, an open-source FTP client. For Filezilla as your GUI, follow these steps to download files:

  1. Open FileZilla and access Site Manager (File > Site Manager).
  2. Create a new connection with the following settings (Figure 1):
    • Protocol: SFTP - SSH File Transfer Protocol
    • Host: outbox.test.ega-archive.org
    • Logon Type: Key file
    • User: your EGA username
    • Key file: path/to/your/private_key

    Figure 1: Process of establishing a new connection to outbox.test.ega-archive.org using a key file as the logon method in FileZilla. The figure showcases the FileZilla version 3.52.2 operating on IOS v11.2.3. By following the depicted steps, users can create a secure and efficient connection to the EGA outbox, ensuring seamless data transfers.

  3. Click Connect to access your Outbox. This folder serves as your storage space within the EGA cloud, containing files accessible for download in a secure way.
  4. Browse the remote directory on the right side of the FileZilla screen. Select the files you wish to download, right-click, and choose Download (Figure 2).

    Figure 2: Step-by-step process of manually downñoad files from outbox.test.ega-archive.org using FileZilla, with FileZilla version 3.52.2 operating on IOS v11.2.3. The figure demonstrates how users can downñoad data from the EGA outbox to their local storage by following the depicted steps

SFTP command line

You can also use the SFTP command line to securely download files from the EGA Outbox.

Using SFTP command line client in Linux/Unix

  1. Open a terminal window
  2. Enter the following command to connect: sftp username@hostname
  3. Enter your EGA password
  4. To see a list of available sftp commands type help
    • sftp> put – Upload file
    • sftp> get – Download file
    • sftp> cd path – Change remote directory to ‘path’
    • sftp> pwd – Display remote working directory
    • sftp> lcd path – Change the local directory to ‘path’
    • sftp> lpwd – Display local working directory
    • sftp> ls – Display the contents of the remote working directory
    • sftp> lls – Display the contents of the local working directory
  5. Type get command to download files. For example: get encrypted_file.c4gh
  6. Use the bye command to close the connection (SFTP session).

Convenient SSH settings

Include the following settings in your SSH config file, located in ~/.ssh/config

Host outbox.test.ega-archive.org EGA-outbox
    hostname outbox.test.ega-archive.org
    User username
    IdentityFile path/to/the/private/key

Replace username and path/to/the/private/key with the appropriate settings, and you will be able to connect to the outbox.test.ega-archive.org simply using sftp EGA-outbox.

How to decrypt

Files archived at the EGA are encrypted based on Crypt4GH. Hence, to decrypt the files you need to install Crypt4GH. You can install a python implementation of it, with

pip install crypt4gh

or directly from the Github repository 

pip install git+https://github.com/EGA-archive/crypt4gh.git

After installing Crypt4GH, decrypt files using the following command:

crypt4gh decrypt --sk /path/private/key < encrypted_file.c4gh > decrypted_filename

The command reads the encrypted file from stdin (with <) and output the decrypted version to stdout (with >).
Replace encrypted_file.c4gh and decrypted_filename with the appropriate filenames but make sure to not use the same filename for both reading and writing because your SHELL would then truncate both files before you even read or write.

Frequently Asked Questions

What username should I use to log in to my outbox?

The authentication process for logging in to the EGA website, as well as accessing your inbox and outbox, requires the use of your username, not your email address. Therefore, if you registered a username different from your email address when creating your EGA account, you must use that username to log in.

If you have forgotten your registered username, please, contact our Helpdesk team for assistance.

I see that some files in my dataset have 'unavailable' as extension. What should I do?

Within your Outbox, you'll find a list of all the datasets available for download. Occasionally, certain files may be marked as "unavailable".

These unavailable files can be identified by the "unavailable" extension added to their filenames (e.g. filename.fastq.gz.unavailable.c4gh).

If you encounter an unavailable file that you need, please reach out to our Helpdesk. We'll promptly work on making the file accessible for download as soon as possible.

Specific to using keys


Can I access one EGA account from different devices?

Yes, you can access your account from different devices by linking several public keys to your EGA account. Each device can generate a unique public-private key pair, and the corresponding public keys can be linked to the same account. This way, you can use different public keys on different devices and still have access to the same account and data.

I have several keys and I don't remember which one is which

When generating SSH keys, it's a good practice to add a comment using the -C flag. This will allow you to add a descriptive tag to your key, making it easier to identify later on. Here's an example command that generates an SSH key with a comment:

ssh-keygen -t ed25519 -C work-pass

In this example, we're generating an ed25519 SSH key with the comment work-pass. Once you have multiple keys with different comments, you can use the comments to easily identify each key.

To view the comments for your existing SSH keys, you can use the following command:

ssh-keygen -l -f /path/to/key

This will display the key fingerprint and the associated comment. By checking the comments, you should be able to identify which key is which.

What if I can't find my SSH keys for uploading files with a key file, and how can I use new keys?

If you can't find your SSH keys, don't worry - you can make new ones. To do this, open your terminal or command prompt and type a command to make a new SSH key. You can pick a name for the key, and choose a password to keep it safe. After making the key, you can add the new key to your account or server where you want to upload files using the key file. This usually involves copying and pasting the key's "public" (e.g. file.pub) part to the right place. If you lose track of the key again, just make a new one and add it again.

Keep in mind that SSH keys belong to you and your computer, so if you switch computers or accounts, you'll need to make new keys.

I don't want to type the passphrase every time I use the key. What can I do?

You can use an ssh-agent to avoid typing the passphrase every time you use the key. An ssh-agent is a program that stores your private keys in memory and provides them to ssh when needed. You can add your key to the ssh-agent using the command ssh-add followed by the path to your key file.Here's an example of the steps to follow:

  1. Open a terminal window.
  2. Start the ssh-agent by typing the command eval $(ssh-agent).
  3. Add your key to the ssh-agent by typing the command ssh-add [key filepath].

For instance, if your key file is located in the home directory with the name mykey, the command will look like this:

ssh-add ~/mykey

After adding your, key to the ssh-agent, you should be able to use ssh without having to enter your passphrase every time.

Can I use my password for authentication (without my private key)?

If you prefer to use your username and password for authentication instead of your private key, you can still do so. When using a Graphical User Interface (GUI) such as FileZilla, you can select Ask for password as your Logon Type (Figure 3). This option will prompt you to enter your password when you click Connect, instead of using your private key.

Figure 3: This option will prompt you to enter your password when you click "Connect", instead of using your private key. Figure 3: Process of establishing a new connection to outbox.test.ega-archive.org using your password as the logon method in FileZilla. The figure showcases the FileZilla version 3.52.2 operating on IOS v11.2.3. By following the depicted steps, users can create a secure and efficient connection to the inbox, ensuring seamless data transfers.

It's worth noting that using a password for authentication can be less secure than using an SSH key, as passwords can be more easily compromised through various means. However, if you choose to use your password for authentication, selecting "Ask for password" as your Logon Type is a good way to do so securely via a GUI.

Why is it better to use my key and not my password?

SSH keys for authentication is generally considered to be more secure and convenient than using passwords. SSH keys are more difficult to crack than passwords, and they can be restricted to specific users and machines, giving you more control over access. Once you set up your SSH keys, you can use them to authenticate quickly and easily, without having to enter a password every time. This makes automation of tasks, such as uploading encrypted files, much simpler. Additionally, SSH keys provide better logging, allowing you to keep track of who is accessing your systems and when. All in all, using SSH keys is a good practice for improving security and convenience in your authentication process.