Ethics of screen scraping
Ethics of screen scraping
Author
Discussion

pugwash4x4

Original Poster:

7,684 posts

252 months

Friday 21st August 2009
quotequote all
Wondered what the general consensus was on using screen scraping software to create mailing lists that you selectively and carefully market to.

just started on a brand new project, selling items which aren't really used in the UK yet (digital signage and large format displays) and i'm struggling to create leads. One option is screen scrape software to pull adrresses and phone numbers from websites, who i would then vet and start to call. What does everyone else think of this- is screen scraping even illegal, and if if not is it morally reprehensible.

The info would go out of my control (ie i wouldn't sell or lease it), and i wouldn't just mass spam everyone. on the other hand i don't want to put potentials clients backs up by making unwanted calls and spam.

Its an odd position because my last 2-3 businesses have been retail only, and we've had all the work we could manage and handle from referrals and personal recommendations> i'm struggling to create leads and its begnning to depress me!

Kermit power

29,622 posts

244 months

Friday 21st August 2009
quotequote all
I don't see why anyone would have a problem with how you obtain the data if it's all public domain stuff on the web. It's how you then use it which makes all the difference.

mrmr96

13,736 posts

235 months

Friday 21st August 2009
quotequote all
You could always buy a mailing list? (It would then be legal and more chance of it being targetted at the people you're after talking to.)

Simpo Two

93,108 posts

296 months

Friday 21st August 2009
quotequote all
I think people have been trying to sell/barter things to/with other people since caveman days. It's how you do it that makes the difference. Ask politely - the worst they can do is say no.

rpguk

4,520 posts

315 months

Friday 21st August 2009
quotequote all
A couple of separate issues here IMHO

In terms of the site you're scraping - If it's for your own use only then I don't really see any difference between doing this and just going through the yellow pages. Any lost revenue for the sites being scraped in this case is negligible. No different to using an adblocker.

From a legal point of view you'll be fine so long as you don't sell the data or whatnot.

In terms of the people you're calling. Well it's standard cold calling really. Not something I'm a fan of or something which is effective to me but hey ho. Only thing to be careful of is that you're not calling people on the TPS.

pugwash4x4

Original Poster:

7,684 posts

252 months

Friday 21st August 2009
quotequote all
rpguk said:
Not something I'm a fan of or something which is effective to me but hey ho. Only thing to be careful of is that you're not calling people on the TPS.
totally agree but it is very very hard to mass market to people on a budget without doing something like email marketing, cold calling etc etc

if anyone has any better ideas then i;d love to hear them!

mrmr96

13,736 posts

235 months

Saturday 22nd August 2009
quotequote all
pugwash4x4 said:
rpguk said:
Not something I'm a fan of or something which is effective to me but hey ho. Only thing to be careful of is that you're not calling people on the TPS.
totally agree but it is very very hard to mass market to people on a budget without doing something like email marketing, cold calling etc etc

if anyone has any better ideas then i;d love to hear them!
Buy a list! The supplier of the list will have been able to cross reference it with TPS etc. (Seems like I'm talking to myself sometimes.)

jamesuk28

2,176 posts

284 months

Saturday 22nd August 2009
quotequote all
Correct me if I am wrong but collecting email addresses and emailing the recipient without them agreeing to it is spamming is it not, and thats illegal whether or not you use the data yourself or sell it on.

There was a case about two years ago where an English company sent a Scottish company an unsolicitored email (spam) the Scottish company sued the English company and won, just found it...

Man Sues and Wins against ISP for Spamming Mail
An Internet service provider has to pay 750 pounds together with expenses and interest to a man it spammed a single e-mail, ordered a Scottish court.

Transcom Internet Services will be refunding Gordon Dick the damages it caused in various sections. It will pay 750 pounds plus an interest amount at 8% since May 2006 plus 618.66 pounds as other expenses. Both the involved parties Transcom and Nominet are, however, disputing about the proceedings of the case that was based on a suit against Transcom ISP for sending Dick only one e-mail. This is the second time Dick has come out victorious in a spam case in UK.

Dick pulled Transcom to an Edinburgh court after he received a marketing e-mail appearing through an e-mail group under U.K. domain registry Nominet. Gordon Dick belonging to Newington reacted after two spam mails came to his private e-mail Id from an English-based firm wanting to sell software.

Dick accused Transcom for harvesting addresses from the Nominet group. But William Smith, Director of Transcom argued that Nominet subscribed Smith's mailbox, which sent promotions every month without taking his permission.

Transcom campaigned its own anti-spam services through its e-mail, which also had an unsubscribe feature. This particular message was delivered to nearly 72,000 people. Smith said Nominet enjoyed the ability to send e-mail directly to recipients and not through its list address, entangling Transcom into the incident.

According to Smith, as part of its regular marketing activities, the company sent the e-mail to Dick. He said every year, his company sent out e-mail making special offers to customers and people who were interested in their business. During all the years of their business, they had received only one complaint, that from Gordon Dick.

Dick charged Transcom in Edinburgh Sheriff Court complaining a violation of the Data Protection Act and the Privacy and Electronic Communications Regulations by sending an undesirable commercial e-mail.

After winning a historic legal triumph over spammers, Gordon Dick, an Internet expert, is preparing to launch a website to help other people battle out unsolicited junk e-mails.


So I guess the advice is be careful.

Taita

8,021 posts

234 months

Saturday 22nd August 2009
quotequote all
mrmr96 said:
pugwash4x4 said:
rpguk said:
Not something I'm a fan of or something which is effective to me but hey ho. Only thing to be careful of is that you're not calling people on the TPS.
totally agree but it is very very hard to mass market to people on a budget without doing something like email marketing, cold calling etc etc

if anyone has any better ideas then i;d love to hear them!
Buy a list! The supplier of the list will have been able to cross reference it with TPS etc. (Seems like I'm talking to myself sometimes.)
Speak to PH'er Don1, I believe he does this (others do it too).

Simpo Two

93,108 posts

296 months

Saturday 22nd August 2009
quotequote all
Gordon Dick. Hmm.

So you can either (a) spend weeks and months of your life writing Mr Angry letters because you got an e-mail about Viagra, or (b) simply delete it and spend week and months of your life enjoying, well, your life.

mrmr96

13,736 posts

235 months

Sunday 23rd August 2009
quotequote all
jamesuk28 said:
Correct me if I am wrong but collecting email addresses and emailing the recipient without them agreeing to it is spamming is it not, and thats illegal whether or not you use the data yourself or sell it on.
Buy a list of people who have agreed to recieve "special offers from our selected partners"

bonsai

2,015 posts

211 months

Sunday 23rd August 2009
quotequote all
mrmr96 said:
jamesuk28 said:
Correct me if I am wrong but collecting email addresses and emailing the recipient without them agreeing to it is spamming is it not, and thats illegal whether or not you use the data yourself or sell it on.
Buy a list of people who have agreed to recieve "special offers from our selected partners"
Ah yes, I always make sure to untick that particular box.

Audicab

496 posts

278 months

Sunday 23rd August 2009
quotequote all
In terms of the legal issues it depends who you want to target.

Consumers
Consumers are defined as private individuals and will have an ISP email address i.e yahoo.co.uk, msn.co.uk etc. For consumer marketing you have to have their permission to email them and you have to be able to prove that they have opted in, there are a few caveats, if you have sold to them since before the legislation changed in 2006 you can still market to them unless they opt-out of future mailings. Even if a company manager uses a Yahoo address for his email you cannot email them.

You also have to be careful about holding personal data and check their phone numbers against the TPS every month.

Businesses
A business is a Ltd, or PLC Not a sole trader or partnership. The email address must be the domain of the company and as such isn't classed as personal data so legally you can collect these emails and email them on an opt-out basis, i.e. you can continue to email them until they say not to, then you have 28 days to stop.

The only confusion comes for parnerships and sole trader who although they use the company name for the domain it is still classed as personal email so works on an opt-in basis.

The emails you send also have to be relevant to the job or company you are sending them to and not personal emails as these are still classed as consumer mailings and are opt-in.

Also for businesses, check the phone number against the CTPS and FPS etc.

As James pointed out there have been a number of court cases but these have been through collecting personal email addresses and communication to them without permission. If you do get caught it is a £750 fine for every email address you communicate to e.g. if you send out 1000 emails it could cost you £750,000 so be very careful.

We do a lot of email marketing and try hard to get businesses to opt in as the emails are so much more successful, but if you are sending speculative business emails ensure that they are relevant and potentially of interest to the job of the people you are sending them to.

Email data is quite cheap to buy though it is very difficult to get hold of good data, we build our lists from scratch and call the companies to ensure it is okay to send the emails to them.

Hope this helps.
Phil