Quantcast
Skip to content

Sharing and Transferring Gigabytes of Data

Sharing and Transferring Gigabytes of Data

Great Scott! 1.21 gigabytes of data!!!

The conundrum is not ‘Where can I store massive amounts of information’, but ‘How can I quickly share massive amounts of information all at once?!’

I’m certain this is a topic we have all wrestled with before, so here are 5 quick ways to send research data out to your colleagues.

1. USB Drives

Where allowed by institution (some prohibit their use due to the ease at which one can walk away with sensitive research and other data), USB flash and hard drives are one of the fastest ways to move data around. With storage sizes ranging from one to hundreds of gigabytes it is science on the go: in the lab, on the road, and even between special equipment that isn’t connected to a network.

Leave antiquated CDs and DVDs behind.

2. Email attachments

Regularly share data with labmates? E-mail is still a favorite method. But be careful: organizations are notorious for setting hard limits on how big file attachments can be. Exceed the limit and an entire message is often deleted without warning! It’s a good idea to check with IT before regularly sending files larger than, say, about 10 megabytes, as a general rule of thumb. For gigs of data, try another method.

3. Network drives

Best for the labs that are separated by great distances on campus, and those souls working the graveyard shifts. Network drives are perfect for sharing data amongst many people because everything is accessible 24/7. Backups are regularly scheduled and automatic. With well organized directories, it’s easy to find everything you need from any PI and any data from any experiment.

4. Secure file servers

A solution to email limits. One can upload data files onto a secure FTP (file transfer protocol) server for a colleague to download later. This option is a little more complicated because it may require use of specialized software, an understanding of transfer protocols, possibly encryption codes and navigation of unfamiliar file servers. However, it is an excellent way to be sure that data remains in-house and always goes to a small group of people.

5. Laboratory Management Systems and Cloud Storage
Cloud, or internet, services can be extremely useful for sharing data within a core group of people because access is always just a desktop or mobile device away. Labs that embrace online laboratory management systems (LMS), like LabGuru for example, instantly benefit from having all of their research and project data stored in a single location. LMS eliminate the frenzied search for locating data that was once stored on various USB or network drives, file servers or e-mail archives and the time it took to distribute it to colleagues. Data is now accessible to everyone on-demand.

Cloud storage acts as a virtual drop box – much like a network drive, or file server at an institution. The biggest difference is that it’s not entirely under your direct control. Cloud storage has the potential to be useful for sharing gigabytes of data, worldwide, from one person to the next, but it’s important to consider where the data will be stored, who can access it, and that the data package doesn’t contain any proprietary or sensitive information.

You’re a scientist, you dabble in data. How large was your largest data set, and what was the most efficient way to share it with your colleagues?

 

 

3 Comments

  1. Nestor on November 12, 2016 at 9:42 am

    I also cast another vote for Dropbox. It offers large amounts of GB storage plus you can use the app available and have your data accessible through your PC, tablet, smartphone, web browser etc. You can set up collaborating teams and specify access levels for each participant as well.
    I agree that e-mail is not the best way to distribute large files, especially in the case where the e-mails are not stored locally, i.e., university e-mail servers, where images of e-mails are kept and large files create a burden on the system.

  2. micronaut on March 11, 2013 at 9:55 am

    dropbox works well for collaborating with Mb’s of data. I once set up ssh tunnel and virtual desktop to work remotely on a machine that was holding 10’s of Gb. It was impossible to transfer that much data between machines.

  3. MichaelA on March 11, 2013 at 8:39 am

    Actually email is the worst way of distributing large files. This is a medium designed to transfer text only, so to attach a file to email, is has to be encoded. This encoding actually makes the amount of data, that needs to be transferred by about 30% bigger, so 3 MB file will generate an email about 4mb in size.

Leave a Comment

You must be logged in to post a comment.

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Scroll To Top
Share via
Copy link
Powered by Social Snap