is there a tool for this web issue?

is there a tool for this web issue?

Author
Discussion

insurance_jon

Original Poster:

4,055 posts

246 months

Tuesday 25th October 2016
quotequote all
say I wanted to monitor specific websites for specific words appearing and get alerts:

i.e this site for the word insurance being mentioned

Is there an app or tool for this?

crmcatee

5,694 posts

227 months

Tuesday 25th October 2016
quotequote all
You can watch for changes on webpages on sites such as this - http://www.watchthatpage.com/index.jsp

It doesn't allow for searching for specific words but it'll alert you if something has changed on it.

murphster

143 posts

258 months

Tuesday 25th October 2016
quotequote all
Might be worth trying google alerts?

https://www.google.co.uk/alerts

Enter [site name] and [search term] ?

Might work?

akirk

5,390 posts

114 months

Tuesday 25th October 2016
quotequote all
google alerts is probably your best bet...
if a website produces its own tool it is quite simple...
otherwise you need to effectively scrape / spider every change to a website and then analyse it - so you would need to store it all somewhere to know what is new / what needs looking at etc. - far more complex...

jonamv8

3,151 posts

166 months

Tuesday 25th October 2016
quotequote all
Google alerts first.

Depending on commercial application then possibly akirks suggestion. Beware some sites monitor scraping and block it

SystemParanoia

14,343 posts

198 months

Tuesday 25th October 2016
quotequote all
Just knock up a cheeky web scraper

https://automatetheboringstuff.com/chapter11/

then have the program fire off an email to you when your conditions are met.
or link it to IFTTT to have an easy phone alert or whatever

Tuna

19,930 posts

284 months

Friday 28th October 2016
quotequote all
First stop is to check to see if the site provides an RSS/news feed. If you're looking for a site that has regular content updates and want to respond to an article, they're likely to have such a feed, and it's not too hard to monitor that.

nyt

1,807 posts

150 months

Friday 28th October 2016
quotequote all
Website watcher is useful for this: http://aignes.com

It continually monitors pages that you configure at a frequency that you set up. If the site changes it'll download the the page. It'll then run a program that you set up to do whatever you like with that data. A program to search for banned words should be easy to write or download.


insurance_jon

Original Poster:

4,055 posts

246 months

Friday 28th October 2016
quotequote all
essentially I am wanting to monitor a few forums for mention of the word insurance

insurance_jon

Original Poster:

4,055 posts

246 months

Friday 28th October 2016
quotequote all
Tuna said:
First stop is to check to see if the site provides an RSS/news feed. If you're looking for a site that has regular content updates and want to respond to an article, they're likely to have such a feed, and it's not too hard to monitor that.
that's gone over my head. can people do this for you?

boony

382 posts

237 months

Sunday 20th November 2016
quotequote all
When I read this post, it got me thinking. I thought that maybe I could build something and charge a monthly fee for the service. If enough people would find it useful then there should be a price point where it’s worthwhile building out the service.

So, I've built a prototype... take a look at : http://fw3.azurewebsites.net/

It’s not fully functional, but it monitors the piston heads business forum for the insurance keyword. It only goes one page deep on each forum, but it’s enough to give you an idea of how it might work.

I'm interested in any feedback...

I’d be interested to know how much value you’d put on something like this. Would you pay for it? If so, how much would you pay per month?


bigandclever

13,787 posts

238 months

Sunday 20th November 2016
quotequote all
boony said:
I'm interested in any feedback...
It amused me (very mildly!) that it picked up a topic where the only mention of 'insurance' was because the OP had posted in it (the Suits thread) smile

boony

382 posts

237 months

Sunday 20th November 2016
quotequote all
me too smile I was thinking of building out User Stalker next smile

insurance_jon

Original Poster:

4,055 posts

246 months

Sunday 20th November 2016
quotequote all
with the right develpment it could have a lot of uses:

1. people like me who respond to set topics on forums
2. PR and Repetitional management for companies

boony

382 posts

237 months

Sunday 20th November 2016
quotequote all
tweaked it so it won't match the keyword in the author section... better results.