Flawed Gadget Show test
Discussion
Tuned last night to see the "music format" test on last night's Gadget Show ? The test was supposed determine if Vinyl or CD were noticeably better sounding than an ipod. Speakers were a pair of Mordaunt Short's top floorstander's I think but didn't see enough of the amp and disc player. The turntable looked pretty poor ~ like a very basic non suspended deck from an old stack system. Did anyone else work out what gear was being used ?
I appreciate that an lossless ipod can sound great and ipods don't really care where they are used but the turntable was on what looked like a wooden table from the canteen and the table was a couple of feet from the speakers. The conclusion that ipod sounded better than the feedback impaired vinyl when playing Pink Floyd's Money loud enough to fill a theater wasn't too much of a surprise.
I appreciate that an lossless ipod can sound great and ipods don't really care where they are used but the turntable was on what looked like a wooden table from the canteen and the table was a couple of feet from the speakers. The conclusion that ipod sounded better than the feedback impaired vinyl when playing Pink Floyd's Money loud enough to fill a theater wasn't too much of a surprise.
The test is about as good as the Top Gear test of finding out whether a pedal bike, a boat or a car is fastest.
Of course the car is fastest, but through central London, all are about the same.
On cheapish amps and speakers there is little difference between iPod sound and CD, or in fact anything better.
Of course the car is fastest, but through central London, all are about the same.
On cheapish amps and speakers there is little difference between iPod sound and CD, or in fact anything better.
I can remember when the purpose of Hi-Fi was to try to get as close to a live performance as possible. When did that idea go out of the window?
Take a quick look at "The Sizzling Sound of Music".
Many years ago, when CAMRA were first making an imapct a daily paper invited a number of people to sample some of the beers on sale at the time, only one was a "real ale". The CAMRA representative voted for the real ale as he could identify that it was but admitted it was not very good, on the day. When a "fizzy", artificial beer won the letters in the CAMRA newsletter were in a similar vein to the comments in this thread.
The test might be flawed but at least they tried a form of blind test which most manufacturers won't touch with a barge pole.
Take a quick look at "The Sizzling Sound of Music".
Many years ago, when CAMRA were first making an imapct a daily paper invited a number of people to sample some of the beers on sale at the time, only one was a "real ale". The CAMRA representative voted for the real ale as he could identify that it was but admitted it was not very good, on the day. When a "fizzy", artificial beer won the letters in the CAMRA newsletter were in a similar vein to the comments in this thread.
The test might be flawed but at least they tried a form of blind test which most manufacturers won't touch with a barge pole.
FlossyThePig said:
I can remember when the purpose of Hi-Fi was to try to get as close to a live performance as possible. When did that idea go out of the window?
Take a quick look at "The Sizzling Sound of Music".
Many years ago, when CAMRA were first making an imapct a daily paper invited a number of people to sample some of the beers on sale at the time, only one was a "real ale". The CAMRA representative voted for the real ale as he could identify that it was but admitted it was not very good, on the day. When a "fizzy", artificial beer won the letters in the CAMRA newsletter were in a similar vein to the comments in this thread.
The test might be flawed but at least they tried a form of blind test which most manufacturers won't touch with a barge pole.
I agree a blind test is the way forward.Take a quick look at "The Sizzling Sound of Music".
Many years ago, when CAMRA were first making an imapct a daily paper invited a number of people to sample some of the beers on sale at the time, only one was a "real ale". The CAMRA representative voted for the real ale as he could identify that it was but admitted it was not very good, on the day. When a "fizzy", artificial beer won the letters in the CAMRA newsletter were in a similar vein to the comments in this thread.
The test might be flawed but at least they tried a form of blind test which most manufacturers won't touch with a barge pole.
However, having the test under conditions where the quality of the amp and speakers do not permit you to actually tell the difference would be like CAMRA having a blind test tasting ale, but before tasting each pint eating an extra strong mint!
You might get some blind results with both tests, but neither will actually prove what you set out to do to try and recreate *actual* conditions.
FlossyThePig said:
I can remember when the purpose of Hi-Fi was to try to get as close to a live performance as possible. When did that idea go out of the window?
You could try turning your system up until it distorts, getting someone to shout at you and spill beer on yourself and you will still get that feeling 
I think of hi-fi as being faithful to the recording, not necessarily to the original performance. However, if a system is too revealing you end up dissatisfied with the majority of recordings so there's room for subjectivity, surely?
SJobson said:
FlossyThePig said:
I can remember when the purpose of Hi-Fi was to try to get as close to a live performance as possible. When did that idea go out of the window?
You could try turning your system up until it distorts, getting someone to shout at you and spill beer on yourself and you will still get that feeling I think of hi-fi as being faithful to the recording, not necessarily to the original performance. However, if a system is too revealing you end up dissatisfied with the majority of recordings so there's room for subjectivity, surely?
The original reason for recording was to capture a live performance.
However, with the advent of multi-track tape machines, recording became an art unto itself, and microphone placement changed from aiming to recreate a 'front row' experience to one with a 'hi-fi' sound, and this has been the ethos ever since.
So I would say the best description of what hi-fi should aim for should be to accurately recreate the recording.
I agree the test (and the show) is completely flawed. However I do partially agree with the outcome. I think audiophiles in general dismiss MP3 without thought, but given a proper, but typicial audiophile home style set up (not necessarily with an iPod) I think anyone would be hard pushed to tell the 3 apart.
It's like wine, all the connoisseurs rave about the bouquet and mouth feel of the top Chateaus, but put in a blind tasting, its often the cheaper vineyards that win out.
At the end of the day it's all completely subjective.
Flameproof suit on.
It's like wine, all the connoisseurs rave about the bouquet and mouth feel of the top Chateaus, but put in a blind tasting, its often the cheaper vineyards that win out.
At the end of the day it's all completely subjective.
Flameproof suit on.
navier_stokes said:
I agree the test (and the show) is completely flawed. However I do partially agree with the outcome. I think audiophiles in general dismiss MP3 without thought, but given a proper, but typical audiophile home style set up (not necessarily with an iPod) I think anyone would be hard pushed to tell the 3 apart.
It's like wine, all the connoisseurs rave about the bouquet and mouth feel of the top Châteaus, but put in a blind tasting, its often the cheaper vineyards that win out.
At the end of the day it's all completely subjective.
Flame-proof suit on.
I dismiss it because I've listened to it on several players. The sound is OK on treble but bass and mid range is compromised to non-existant. Where bass does exist it tends to be of the "one note" variety. In essence the sound is not as "full" as other sources.It's like wine, all the connoisseurs rave about the bouquet and mouth feel of the top Châteaus, but put in a blind tasting, its often the cheaper vineyards that win out.
At the end of the day it's all completely subjective.
Flame-proof suit on.
navier_stokes said:
I agree the test (and the show) is completely flawed. However I do partially agree with the outcome. I think audiophiles in general dismiss MP3 without thought, but given a proper, but typicial audiophile home style set up (not necessarily with an iPod) I think anyone would be hard pushed to tell the 3 apart.
It's like wine, all the connoisseurs rave about the bouquet and mouth feel of the top Chateaus, but put in a blind tasting, its often the cheaper vineyards that win out.
At the end of the day it's all completely subjective.
Flameproof suit on.
Strangely enough, I did this test last night, but not as a test I just listened to a couple of songs that I had to download as they were iTunes store only.It's like wine, all the connoisseurs rave about the bouquet and mouth feel of the top Chateaus, but put in a blind tasting, its often the cheaper vineyards that win out.
At the end of the day it's all completely subjective.
Flameproof suit on.
My setup for both was identical, my WAVs from my CDs were coming from my Mac by optical as were the MP3s.
The MP3s have no bass extension. Further to that and more importantly the sound was very 'closed' in the sense that it almost sounds like you are listening to music in a 'dead room' where the high frequency information which is so vital for the ear to position where instruments are coming from, and give you a sense that sounds are not coming from just left and right, but outside the width of the speaker and are set in front and behind are almost completely lost.
I would agree that it is an excellent format for mobile listening, and indeed on in the ear headphones these type of things are just not an issue - you couldn't hear the difference anyway.
However, I have no requirement for mobile listening or through headphones, so it is pointless for me.
It is also worth remembering that throughout the history of music recording the technology available has altered music permanently. Throughout that time these have always been with the ethos of a benefit to either the artistic qualities or the fidelity of the music.
The ethos of MP3 was to change the game under particular constraints to make a song downloadable and portable. The cost was the audio fidelity and generations to come thinking that is what music 'sounds like'. This was a viable trade when you could only download at 5k a second and it cost a fortune to carry a few albums digitally. However today I can download a full CD album at 5MB a second, and could carry my whole music collection easily on a hard drive.
The effect on recording and music will be there for ages after the technological constraints have been lost however. IMHO, I would rather see the death of MP3.
FlossyThePig said:
The test might be flawed but at least they tried a form of blind test which most manufacturers won't touch with a barge pole.
However the test showed a fundamental lack of understanding of how vinyl works mechanically. A half decent table or some form of isolation should have been used for the turntable and CD ( I'm not talking about Mana Table here by the way ). It would have been hard to select a worse wooden table for the vinyl; naieve at best. I'm not a vinyl die hard and haven't listed to vinyl at home for years now but for the Gadget Show to promote this a fair blind test of the formats was nonsense.
EDIT. Ref your comment about manufacturers not touching blind tests with a barge pole. It may not happen much anymore but I'm old enough to remember when Hi-Fi Choice mag used a panel of representatives from manufacturers to carry out all their testing ~ double blind tests too. Clearly you could be praising the opositions products or be equally damning to your own. Dunlavy Audio famously used to compare their speakers with live musicians in the same auditorium; invited members of the public allegedly could not hear the difference. Can't get much fairer than that.
Edited by CRACKIE on Thursday 19th March 09:48
JustinP1 said:
navier_stokes said:
I agree the test (and the show) is completely flawed. However I do partially agree with the outcome. I think audiophiles in general dismiss MP3 without thought, but given a proper, but typicial audiophile home style set up (not necessarily with an iPod) I think anyone would be hard pushed to tell the 3 apart.
It's like wine, all the connoisseurs rave about the bouquet and mouth feel of the top Chateaus, but put in a blind tasting, its often the cheaper vineyards that win out.
At the end of the day it's all completely subjective.
Flameproof suit on.
Strangely enough, I did this test last night, but not as a test I just listened to a couple of songs that I had to download as they were iTunes store only.It's like wine, all the connoisseurs rave about the bouquet and mouth feel of the top Chateaus, but put in a blind tasting, its often the cheaper vineyards that win out.
At the end of the day it's all completely subjective.
Flameproof suit on.
My setup for both was identical, my WAVs from my CDs were coming from my Mac by optical as were the MP3s.
The MP3s have no bass extension. Further to that and more importantly the sound was very 'closed' in the sense that it almost sounds like you are listening to music in a 'dead room' where the high frequency information which is so vital for the ear to position where instruments are coming from, and give you a sense that sounds are not coming from just left and right, but outside the width of the speaker and are set in front and behind are almost completely lost.
I would agree that it is an excellent format for mobile listening, and indeed on in the ear headphones these type of things are just not an issue - you couldn't hear the difference anyway.
However, I have no requirement for mobile listening or through headphones, so it is pointless for me.
It is also worth remembering that throughout the history of music recording the technology available has altered music permanently. Throughout that time these have always been with the ethos of a benefit to either the artistic qualities or the fidelity of the music.
The ethos of MP3 was to change the game under particular constraints to make a song downloadable and portable. The cost was the audio fidelity and generations to come thinking that is what music 'sounds like'. This was a viable trade when you could only download at 5k a second and it cost a fortune to carry a few albums digitally. However today I can download a full CD album at 5MB a second, and could carry my whole music collection easily on a hard drive.
The effect on recording and music will be there for ages after the technological constraints have been lost however. IMHO, I would rather see the death of MP3.
I would point out though that iTunes downloaded MP3's are only 128kbps, the quality of which is definitely noticeable compared to CD.
In my experience, you need to be at least over 192 kbps for them to become comparable.
Obviously different kit will minimise or maximise the difference though.
Gassing Station | Home Cinema & Hi-Fi | Top of Page | What's New | My Stuff