SW Developers - Databases

[0000]

Efbe said:

can tell there aren't any developers actually answering this, because...

how much data do you have?
What is too big?

I can answer it. 3/5s of FA if they're even trying to use Excel.

[ATG]

Also it sounds like they are just using Excel to analyse the data, and the data itself is already stored in some sort of database. That makes it rather less likely that Postgres, Mongo or any other database is going to be much help, since they'll just replace the data store without bringing any analytical tools to the table.

In general you should be summarising the data to some degree before it gets loaded into Excel. It would be surprising if the existing database was incapable of filtering, aggregating and pivoting its data, so I'd hazard a guess that using the existing tools better might be the easiest solution.

[bitchstewie]

What's the data?

As in type?

I'd be slightly worried if I thought someone's husband was asking a bunch of strangers on the internet where to stick a bunch of medical research data.

Usual bad stuff about default passwords and just forgetting you've stuck a copy here and there and everywhere "for testing" and so on.

Not to mention you'd hope their own IT department wouldn't simply be saying "Sure, do what the hell you want, we don't mind".

[iaindavidson]

Most (not sure if all) NHS trusts have access to a tool called Signals from Noise (No affiliation with them at all) . This is pretty good at analyzing large datasets. If you can get the data into an SfN instance you're only limited by your understanding of statistical analysis.

[Vaud]

Efbe said:

can tell there aren't any developers actually answering this, because...

how much data do you have?
What is too big?

We know potential solutions, we just don't know the exact problem...

[marshalla]

Vaud said:

Efbe said:

can tell there aren't any developers actually answering this, because...

how much data do you have?
What is too big?

We know potential solutions, we just don't know the exact problem...

And recognise the type of customer.

[TurricanII]

OP, try advising the missus to click File > Save As, and save in Excel Binary or XLSB format. This will reduce the file size and generally seems to improve the handling of large files:

https://blogs.msdn.microsoft.com/dmahugh/2006/08/2...