How can I upload 1TB of data to the cloud?
How can I upload 1TB of data to the cloud?
Author
Discussion

dibbers006

Original Poster:

14,640 posts

249 months

Monday 30th July 2018
quotequote all
The initial message was deleted from this topic on 15 December 2023 at 22:17

illmonkey

19,842 posts

229 months

Monday 30th July 2018
quotequote all
Even at ridculously slow speeds, it shouldn't take that long. 1Mbps at full chat will be 3 months, and that is slow!

What speeds do you get?

Do you not know anyone who can take it to work and upload or who is on fibre?

A computer shop may let you dump your PC in there and upload on a faster connection...

Nigel_O

3,903 posts

250 months

Monday 30th July 2018
quotequote all
Find someone with a hot Virgin Media connection, although their upload speed isn't too clever

The work environment is a better idea, as many bigger offices will have much better upload speeds

What is your current upload speed?

Gluggy

711 posts

140 months

Monday 30th July 2018
quotequote all
Nigel_O said:
Find someone with a hot Virgin Media connection, although their upload speed isn't too clever

The work environment is a better idea, as many bigger offices will have much better upload speeds

What is your current upload speed?
Our upload speed on Virgin is around 12mbs so around 200 hours to transfer it - 10 days doesn't sound that bad but only snag is the upload restrictions during peak times so piratically going to take 15-20 at days at least.

Agree on faster upload speeds in a office but can't see the sysadmin being too keen having a "foreign" drive connected to their network or that much data being moved around in one hit....

illmonkey

19,842 posts

229 months

Monday 30th July 2018
quotequote all
Gluggy said:
Nigel_O said:
Find someone with a hot Virgin Media connection, although their upload speed isn't too clever

The work environment is a better idea, as many bigger offices will have much better upload speeds

What is your current upload speed?
Our upload speed on Virgin is around 12mbs so around 200 hours to transfer it - 10 days doesn't sound that bad but only snag is the upload restrictions during peak times so piratically going to take 15-20 at days at least.

Agree on faster upload speeds in a office but can't see the sysadmin being too keen having a "foreign" drive connected to their network or that much data being moved around in one hit....
But when you are the sysadmin... hehe

Currently have a 1Gbps leased line thats only routed for my PC. Wouldn't take long on that type

markiii

4,342 posts

225 months

Monday 30th July 2018
quotequote all
slowly

sjg

7,690 posts

296 months

Monday 30th July 2018
quotequote all
Can you pull from another cloud data source that does do drive shipping? Microsoft charges less than £60 to get data from a shipped drive into Azure storage, then you just pay the usual fee to store it there (under 6p/GB/month). I think Google and Amazon charge similar.

survivalist

6,124 posts

221 months

Monday 30th July 2018
quotequote all
sjg said:
Can you pull from another cloud data source that does do drive shipping? Microsoft charges less than £60 to get data from a shipped drive into Azure storage, then you just pay the usual fee to store it there (under 6p/GB/month). I think Google and Amazon charge similar.
That initially looked cheap, but for 1 Tb that’s roughly £60 a month, every month in perpetuity. Unless you’re actually accessing on a regular basis that’s a pretty expensive way to
Store data. Assuming it’s personal stuff anyway.

Gluggy

711 posts

140 months

Monday 30th July 2018
quotequote all
survivalist said:
That initially looked cheap, but for 1 Tb that’s roughly £60 a month, every month in perpetuity. Unless you’re actually accessing on a regular basis that’s a pretty expensive way to
Store data. Assuming it’s personal stuff anyway.
Think the idea was to pay MS to copy the drive to their cloud and then copy from there to the current provider using their massively faster connections / infrastructure - once copied cancel the MS cloud account to avoid further charges.

bloomen

10,299 posts

190 months

Tuesday 31st July 2018
quotequote all
Make it a nightly ritual before bed split between a few people with half decent connections and it'll happen eventually.

Odd that there aren't a few dedicated uploading companies considering how agonising it can be. 1 tb would take my internet several years and make it unusable throughout.

Edited by bloomen on Tuesday 31st July 09:29

peterperkins

3,459 posts

273 months

Tuesday 31st July 2018
quotequote all
1TB is a lot of stuff, can we ask what it is?
Photos? Documents? Plans for world domination?

Have you done some pre-work on your archive like searching for and removing duplicate or end of life files.
Have you done zip or equivalent compression on the files?
Do you really need remote access to it via the internet?

Or is it better just to get another/several 1tb drives, copy it and give it to a trusted family member to keep in a draw at another location?

After doing all the above, if you still want to upload it, just setup a dedicated machine to sit on your ISP or a friends faster service and leave it chugging away at it.

seveb

308 posts

104 months

Tuesday 31st July 2018
quotequote all
Do you know somebody who works at a data centre ? They will have access to at least 1Gbit/s upload from their office, a lot more if they're not limited by typical gig access switches.

Can you de-dupe and compress it ? Something like Veeam backup can do that for you.

Otherwise just leave it going in smaller chunks, say 50GB at a time.

NDA

25,847 posts

256 months

Wednesday 1st August 2018
quotequote all
Quicker to post it. smile

TonyRPH

13,554 posts

199 months

Wednesday 1st August 2018
quotequote all
According to calctool.org if you have a typical upload speed of around 12 Mbps (mega bits per second) uploading 1TB should take around 176 hours or 7.3 days.

You don't say what upload speed you have, but if it's anywhere near the above, I would just set it uploading.

Did your cloud backup provider supply you with any specific software to perform the backup? Usually said software will be able to cope with outages etc. and just resume the upload once back online.

EDIT: The above assumes ideal conditions of course. But there's unlikely to be a huge variation over a 7 day period.

Edited by TonyRPH on Wednesday 1st August 09:09

devnull

3,873 posts

188 months

Saturday 4th August 2018
quotequote all
You don’t know anyone in university, do you? Campuses on site usually have gigantic internet connections that are so quick they make even modern laptops wheeze under some data transfer speeds.

Speaking as someone who utterly rinsed the compsci labs internet during my time there

GetCarter

31,342 posts

310 months

Saturday 4th August 2018
quotequote all
My upload speed is 0.31 mbps. Guess it would take me a while. wink

ian in lancs

3,853 posts

229 months

Saturday 4th August 2018
quotequote all
TonyRPH said:
According to calctool.org if you have a typical upload speed of around 12 Mbps (mega bits per second) uploading 1TB should take around 176 hours or 7.3 days.

You don't say what upload speed you have, but if it's anywhere near the above, I would just set it uploading.

Did your cloud backup provider supply you with any specific software to perform the backup? Usually said software will be able to cope with outages etc. and just resume the upload once back online.

EDIT: The above assumes ideal conditions of course. But there's unlikely to be a huge variation over a 7 day period.

Edited by TonyRPH on Wednesday 1st August 09:09
A while ago I used a 55/18 Mbps connection to upload a 2Tb HDD to the cloud. Carbonite was the host. Mostly photographs in raw and various formats. I left my computer uploading for three weeks. I concluded it would take 18months 24/7 to complete and if I continued to shoot/edit at the rate I was it would never catch up. I scrapped that idea, went for local raid setup and monthly HD backups stored off site.

Cloud is conceptually better but not practical for non-business users yet.


Edited by ian in lancs on Saturday 4th August 21:46

TonyRPH

13,554 posts

199 months

Saturday 4th August 2018
quotequote all
ian in lancs said:
A while ago I used a 55/18 Mbps connection to upload a 2Tb HDD to the cloud. Carbonite was the host. Mostly photographs in raw and various formats. I left my computer uploading for three weeks. I concluded it would take 18months 24/7 to complete and if I continued to shoot/edit at the rate I was it would never catch up. I scrapped that idea, went for local raid setup and monthly HD backups stored off site.

Cloud is conceptually better but not practical for non-business users yet.


Edited by ian in lancs on Saturday 4th August 21:46
Indeed, the figures cited are under ideal conditions, and assumes that the bandwidth is available on the far end as well.

Anecdotally, my son had a server in a datacentre for a while, with 1GB symmetrical connections, and he uploaded some backups on behalf of a friend - however the remote end limited the incoming bandwidth to something silly like 25 megabits /s or so...

I seem to recall he was given about 6TB to upload - it took months....

So having a super fast upload was pointless.

I guess some of these cloud companies do this on purpose, to dissuade users from using the full space allocation.


WindyCommon

3,938 posts

270 months

Saturday 4th August 2018
quotequote all
In a similar situation once, I spoke with my ISP who let me go to their premises and run the upload using their T1 backbone (or whatever it was called) connection. One of the advantages of using a small commercial ISP over the mainstream behemoths I suspect....

Rawwr

22,722 posts

265 months

Friday 10th August 2018
quotequote all
GetCarter said:
My upload speed is 0.31 mbps. Guess it would take me a while. wink
F*ck me! It'd be quicker to call the provider on the phone and read your data to them in binary.