How to Import and Export Data in TSD

Important Information

File naming conventions

It is important that the file name will respect few important rules:

  1. First character can be any number, character, hyphens or underscores excluding "." (namely 0-9a-zA-Z_- )
  2. Subsequent characters may be any number, character, hyphens or underscores including "." (namely 0-9a-zA-Z_-. )
  3. You can use the Norwegian characters æøåÆØÅ
  4. You can not use spaces, other special characters or other alphabets (cyrillic, etc.).

How to import and export data using the web file transfer

You can import files from your web browser using the web file transfer service. Read more about it here.

Importing Data

The imported data is stored in file-import  folder of the durable volume of your project - /tsd/pXX/data/durable/file-import/pXX-member-group.

Exporting Data

The exported data is read from the file-export folder of the durable volume of your project - /tsd/pXX/data/durable/file-export.


Please use updated version of the internet browser. Google Chrome and Firefox are preferable. In order to use the service you need to have the javascript and cookie enabled.


How to Import/export non-encrypted sensitive data

We have designed the FX03 solution to allow import and export of non-encrypted sensitive data from sites where there is a control of the network from end to end. We have established FX03 solution with the following end-sites:

  • OUS - AMG/NSC Ullevål     (ull-gen-c2960g port 19 og 20)
  • OUS - Radium                     (radium-c2960g  port 1-4
  • UIO - NSC                           (bioe-9-sw port 12)

To set up the connection initially

  1. Check that you have the folders /fx/import_alt/ and /fx/export_alt in your TSD file system. These are the folders connected to fx03.
  2. Find the MAC adress of the server(s) you wish to hook up and send these MAC adresses to
  3. After getting the OK from TSD, please hook up your server to the correct port number in the switch and try connecting sftp to tsd-fx03.tsdintern.
  4. You can now transfer files, preferrably by creating a Tar/Zip file first.

SSH fingerprint

  • An important part of the security of FX03 data transfer is that the traffic is sftp encrypted between the project and TSD.
  • The ECDSA key fingerprint is ae:bc:70:e2:1c:d2:80:ee:e4:3b:5e:2a:51:1d:60:6f.
  • If projects see other fingerprints, please contact and stop any transfer of data.

How to delete files from Export folder

  • Log in to your TSD project's VM.
  • Find the export directory (pxx/fx/export).
  • Find the files you have exported and delete them.
  • Login via SFTP/WinSCP/Filezilla, and find your project directory (pxx)
  • Go to the "export" directory and delete the files
  • If you delete first from outside, then the files will be copied again. Please be sure that your files are copied before deleting

How to use command line REST-API based data import

For researchers who prefer working on the command line, it is possible to use the same upload service with the TSD API client. There is no limitation on the size of data uploads when using the TSD API client. Files larger than 1 gigabyte will automatically be resumable. If for some reason your current upload fails, then you can resume it without having to upload the same data more than once.

For researchers who need to incrementally synchronise datasets into TSD, we recommend the S3 API. This is an object storage service, compatible with Amazon Web Services' S3 API. This service, which is available on request, and allows uploads with restart capabilities, and incremental synchronisation.

You can download and install the latest version of the command line client for both the file API, and the s3 API here.

How to encrypt files

Use 7-zip

This is only for TSD users using Windows on the inside and outside of TSD:

  1. Download and install 7-Zip.
  2. Start the 7-Zip program.
  3. Disconnect your computer from the internet (the best is to disable Wifi and unplug the internet cable).
  4. Attach the storage with your sensitive data.
  5. Add files into the 7-Zip.
  6. Click next.
  7. Click encrypt files (this automaticly writes files back to the same directory as the raw data was). NB! Be sure to tick off "Do not erase original data".
  8. Store encrypted files on computer somewhere that you are able to find later.
  9. Remove all attached storage devices with sensitive data, and sensitive files you stored on the computer before encryption.
  10. Connect to the internet, use the standard TSD "fil-sluse" to import data into TSD.
  11. Log in to TSD.
  12. Double-click your 7-Zip files.
  13. Type in your password and press decompress.
  14. Your files are decrypted.

Please have a look at the illustrated Norwegian Version.

Troubleshooting and known issues

When an error occurs during transfer, the error message is reported into a file named tsdfx-error.log located in the destination directory (es. /tsd/pXX/fx/import). This file will contain the most recent errors detected for a given set of source destination directories, and is updated every time a new error is detected. Typical errors reported here are files with names containing invalid characters, files with invalid file permissions, symlinks and dot files. Such errors should be fixed in the source directory for the copying to take place. The tsdfx-errors.log file can be safely removed, as it will be recreated when a new error is detected.

Normally problems occur when:

  • the filename contains wrong characters
  • the user trying to export does not have export rights
  • users are trying to import/export files with invalid file permissions, symlinks and dot files. Such errors should be fixed in the source directory for the copying to take place.
  • the import or export folder in the project is full. You can check the usage here.
Did you find what you were looking for?
Published June 30, 2016 4:15 PM - Last modified May 7, 2019 11:41 AM