Webste idea feedback
Webste idea feedback
Author
Discussion

chickenboxprick

Original Poster:

876 posts

216 months

Tuesday 10th September 2013
quotequote all
The initial message was deleted from this topic on 19 January 2014 at 16:36

Cunning Punt

486 posts

184 months

Tuesday 10th September 2013
quotequote all
Hope you're good at regex.

Unix diff will help, if you can reliably strip all the HTML out.



Nice idea, good luck with it thumbup


The Moose

23,710 posts

240 months

Tuesday 10th September 2013
quotequote all
Why would business users not just sign up for a free account and monitor their competitors through that?

LooneyTunes

9,541 posts

189 months

Tuesday 10th September 2013
quotequote all
What percentage of consumers actually care about policies/Ts&Cs to even read them in the first place?

The business angle is interesting, but could be particularly valuable if you were also able to give an option to highlight bigger changes to the sites (e.g. through comparison of sitemaps?).

LooneyTunes

9,541 posts

189 months

Tuesday 10th September 2013
quotequote all
chickenboxprick said:
[1] I think an emerging trend is that the general public are slowly becoming aware of these policies and their importance, more so in the states. Also alot of businesses use social media too, and the policies can have implications for them too. I would probably add ebay policies to the list too.

[2] The business side will monitor webpages individually, say up to 5 for free.
But yes, even if more harder and hence more expensive, monitor a whole website. However intially would like to keep the idea simple, limiting to just a manual entry of webpages by the user.
Re [1]: they may be becoming more aware but if you've signed up then you'll usually get an email if there's a change. Anyone who cares enough to read them in the first place will care enough to read the update.

Re [2]: I'd say that my suggestion isn't much more intensive. All you've got to do is cache the sitemap and then compare periodically. Probably less work than caching/comparing the content on a regular webpage, but potentially higher perceived value?

LooneyTunes

9,541 posts

189 months

Tuesday 10th September 2013
quotequote all
chickenboxprick said:
I think the idea of scanning a whole site is more fuller and compelling, so yes i agree with you, and i can do that. of course not all sites have a sitemap (sigh), but yes its not that hard to compare a whole site for chanages if there isn't hundreds and hundreds of pages.
Start with those that do have sitemaps... will be simpler and require less caching. Find a way to charge more for those that don't (without referencing that you'd otherwise use sitemaps)? Not worth trying to boil the ocean, especially if you've not done your market research to check market demand and potential pricing levels.

bishbash

2,447 posts

228 months

Wednesday 11th September 2013
quotequote all
Sounds like a good idea to me, If I was going to do it, I'd probably pick about 20 to 40 sites from the top 100 sites on Alexa.com and allow people to select from them for free. If they want to monitor anything custom they pay, something like $1 a month per site. I don't think that's unreasonable.

Frimley111R

19,436 posts

265 months

Wednesday 11th September 2013
quotequote all
Apologies for the slight thread drift but is there a site where you can ask it to keep looking at another site and let you know when something you are looking for is posted up? For example, if I wanted a red ford fiesta from my local dealer could I set up a site to check it every few minutes and alert me as soon as a red ford fiesta is 'posted'? This thread about checking sites reminded me of the way it could work potentially.

jammy_basturd

29,778 posts

243 months

Wednesday 11th September 2013
quotequote all
Frimley; you should be able to set that up with Google Alerts.


I would agree with LooneyTunes, not sure people care enough about T&Cs. I mean I had an email from PayPal yesterday about them updating their contract. Deleted it straight away - not like I'm going to stop using their service whatever those changes are. If any of the big sites changed their T&Cs in such an unfair way it would crop up on some of the news/blog sites I visit.

I'd suggest finding X number of people who say they'd use your service, where X is the minimum number of people you need to make this business plan viable.

KFC

3,687 posts

161 months

Wednesday 11th September 2013
quotequote all
chickenboxprick said:
Additionally for business users (there will be a fee), you'll be able to monitor a competitors website and be emailed each time they change something with the exact change highlighted.
How do you propose getting around website t&c's that say no commercial bots/scraping allowed? Or are you just intending on ignoring that.

There is no way I'd want your bot eating up bandwidth I paid for, to feed data to one of my competitors.


marshalla

15,902 posts

232 months

Wednesday 11th September 2013
quotequote all
Doesn't this do much the same thing ? http://www.watchthatpage.com/

KFC

3,687 posts

161 months

Wednesday 11th September 2013
quotequote all
chickenboxprick said:
interesting and fair point, if that was the case, then perhaps it would be easier to compare the google cache results instead, as that wouldn't be toching their site and in the public domain?
It would only move the problem slightly - you'll now be breaking googles t&c's and will quickly find your IP's blocked if you do it to any sort of scale.

KFC

3,687 posts

161 months

Wednesday 11th September 2013
quotequote all
chickenboxprick said:
interesting and fair point, if that was the case, then perhaps it would be easier to compare the google cache results instead, as that wouldn't be toching their site and in the public domain?
It would only move the problem slightly - you'll now be breaking googles t&c's and will quickly find your IP's blocked if you do it to any sort of scale.

jammy_basturd

29,778 posts

243 months

Wednesday 11th September 2013
quotequote all
chickenboxprick said:
marshalla said:
Doesn't this do much the same thing ? http://www.watchthatpage.com/
kinda, yes. obviously there will always be competiton, but their website is quite old looking and I can think of ways id do mine slightly different. however they've been going since 2001 so there clearly is a demand, which is a good thing.
I think it would be a mistake to find one other website offering a similar-ish service and therefore assume there would be a demand for your idea. By your own admission that website doesn't look like it's been updated in a long while. It's just as likely that the creators of that site built it on the assumption there would be demand, there wasn't (comparatively) and so it costs them nothing to just keep it going but never update it.

jammy_basturd

29,778 posts

243 months

Wednesday 11th September 2013
quotequote all
Good thinking. Have you been reading up about Lean Startup then?

Personal

10 posts

158 months

Wednesday 11th September 2013
quotequote all
OP: Does your idea rely on the T&Cs being text on a webpage, i.e. would it be flummoxed by the information being shared in pdf format, for example?

KFC

3,687 posts

161 months

Wednesday 11th September 2013
quotequote all
Personal said:
OP: Does your idea rely on the T&Cs being text on a webpage, i.e. would it be flummoxed by the information being shared in pdf format, for example?
It would need to be somewhere that was publicly shared and could be accessed automatically - a .pdf file that was hosted online and not password protected would be readable just like a regular web page.

I'm not really convinced in the business model though - Anyone with a commercial need for it (scraping competitors prices etc) will need to keep it in house due to the rather dubious use of other peoples web servers.

Sure you might get some freebie accounts wanting to see if Facebook change their terms... but you aren't making any money from those. I can think of at least a dozen better ideas for websites than this one, its just a complete non starter for me.

Frimley111R

19,436 posts

265 months

Wednesday 11th September 2013
quotequote all
jammy_basturd said:
I think it would be a mistake to find one other website offering a similar-ish service and therefore assume there would be a demand for your idea. By your own admission that website doesn't look like it's been updated in a long while. It's just as likely that the creators of that site built it on the assumption there would be demand, there wasn't (comparatively) and so it costs them nothing to just keep it going but never update it.
Agreed. However, the success of the site is based on marketing as much as anything else. Your site may be awesome but if no-one knows its there... I am sure plenty of people came up with the same concept as Moneysupermarket. However the people behind that site spent st loads on marketing whereas others didn't.

jammy_basturd

29,778 posts

243 months

Wednesday 11th September 2013
quotequote all
Rework! is one of my 'goto' books, along with Code Complete. I have Joel On Software's book too, but most of that is already on his blog.

Haven't yet read The Star Principle so I'll look that up, thanks for the recommendation.