Server security/hacking issues

[judas]

Not sure if this should go in Business or Computer forum, but as it's IT related I'll stick it here.

Due to a monumental cock-up by our server hosting company, our web server's firewall has been in "pants down, hands grasping ankles" mode for a while, and during that time we've had some uninvited guests come play. The firewall's been locked down again and we've cleaned up the server as best we can and are still investigating some other issues, but we've come to the conclusion we need some outside help on the security side of things - none of us are what you'd call proper system or network admins.

Need to find one or more people in the East Midlands area who can advise us on the following:

- Windows Server 2003 admin/security, particularly IIS
- MySQL security
- Dealing with Server/Scripting vulnerabilities

Would be good if they didn't charge the GDP of a small country per hour too

Anyone here into this side of things, or any recommendations?

[PetrolTed]

YHM

[JamieBeeston]

I don't know what Budget you are running, but a Checkpoint Firewall solution would handle most of those issues in itself.

It has Stateful Inspection... allowing for full inspection of packet Data, not just the headers like regular firewalls.

They can be set to auto-detect and block malicious IIS exploits (like Code Red / Nimbda) SQL Injection attempts, Ping of Death and DDOS attacks.. but also Security Upgrade features like SSL (upgrade on the fly the protocol of SSL used to communicate with clients.. you use V2 on server, it send out in V3 and translates back)

But these things aren't cheap...

If you're budget is low (low 5 figures and under) then I would point you to a security consultant.

Shame on your host tho..

feel free to drop me a mail via profile of you want to discuss your requirements off board.

J

[tvrforever]

these guys look at many of the FTSE companies and I've trusted them with my reputation & job many times :-

www.integralis.co.uk/

One of the best teams around - looked after my previous global retail www sites.

[judas]

Cheers chaps! We're only a small company with limited budget, so five figure firewalls are definitely out We pay for the firewall as part of our hosting package, and had it been working () we could have probably coped with any issues as they arose. But this just left us far too exposed and we're far too busy trying to keep the business running to devote the necessary resources to dealing with this.

[_Dobbo_]

We use reverse proxy servers to protect any vulnerable systems from the internet. So whilst IIS and coldfusion may be risky apps to run, because the reverse proxy handles all the requests, there are limited things any hacker could do.

This is a cheap and cheerful solution and works very well for us!

[rico]

The whole idea of a firewall is to protect your server. If it goes down its freaking useless.

Speak to Jamie at Serverstream. Your current host sounds crap...

[off_again]

judas said:

Would be good if they didn't charge the GDP of a small country per hour too

That puts my company out then....oooppsss...

[_DeeJay_]

judas said:

Not sure if this should go in Business or Computer forum, but as it's IT related I'll stick it here. Due to a monumental cock-up by our server hosting company, our web server's firewall has been in "pants down, hands grasping ankles" mode for a while, and during that time we've had some uninvited guests come play. The firewall's been locked down again and we've cleaned up the server as best we can and are still investigating some other issues, but we've come to the conclusion we need some outside help on the security side of things - none of us are what you'd call proper system or network admins. Need to find one or more people in the East Midlands area who can advise us on the following: - Windows Server 2003 admin/security, particularly IIS - MySQL security - Dealing with Server/Scripting vulnerabilities Would be good if they didn't charge the GDP of a small country per hour too Anyone here into this side of things, or any recommendations?

It sounds like you're attempted to clean up the mess created, is that right?

I'd recommend you consider rebuilding the server from scratch. Once it has been compromised I'd never trust a server again (they're sneaky buggers those hackers and there will be something you've missed).

If the hosting company cocked up and you use a single dedicated server it wouldn't be too much to ask for them to provide you with another machine so that you can swap over and avoid lots of downtime.

D

[tvrforever]

Oh I can also fully recommend www.mistral.net as a hosting ISP - they host some of the top sites and the guys there really know their stuff...

[JamieBeeston]

_DeeJay_ said:

It sounds like you're attempted to clean up the mess created, is that right? I'd recommend you consider rebuilding the server from scratch. Once it has been compromised I'd never trust a server again (they're sneaky buggers those hackers and there will be something you've missed).

I would agree to an extent, typically its not worth recovering, and a Reformat / Restore is the preferred route, however on some machines / setups, this is just too costly (data and service loss) and so it pays to have the machine professionally detailed and cleansed.

_DeeJay_ said:

If the hosting company cocked up and you use a single dedicated server it wouldn't be too much to ask for them to provide you with another machine so that you can swap over and avoid lots of downtime. D

I would agree here, our policy is certainly 'if we break it, we fix it for you' If you paid someone to maintain a firewall, and they didnt, i'd be playing merry hell with them, and at the least demanding your money back, plus billing for time spent cleaning the system.

I do however accept that diffierent MSPs charge less, and as such provide less. Horses for Courses.

[judas]

JamieBeeston said:

I would agree here, our policy is certainly 'if we break it, we fix it for you' If you paid someone to maintain a firewall, and they didnt, i'd be playing merry hell with them, and at the least demanding your money back, plus billing for time spent cleaning the system.

We have a control panel for maintaining the firewall ourselves, but the host had screwed up the control panel system and the rules weren't being applied to the firewall. Not only that but we were having terrible slowdown problems with server access - this was supposedly tracked down to a faulty firewall, which was replaced and left wide open by default and we couldn't use the control panel to close it, and the support drones just didn't want to know. All they kept saying was RTFM, even though the FM was wrong and the control panel didn't work.

[JamieBeeston]

Thats not good.

If there is an issue, resolve it first, play the blame game second...

Hope you get it sorted.. never fun recovering a compromised server...

Feel free to drop me a mail if you want a chat.

J

[judas]

Update: we've caught one of the hackers!

One of the sites on the hacked server is for a school and it was the site causing us the most problems. We checked the referrer logs and found several link-throughs from a hacking site. So we started reading the forums and some idiot who freely admitted being a pupil at the school was posting stuff about the site/server and other rather incriminating information. Not only that but he'd already e-mailed us over his 'concerns' about the server being slow! Long story short, left it a while to gather evidence while the school was away for half term, then gave the head teacher a call...

Apparently he was dragged out of a lesson and told to come clean or face the very dire consequences. When he got home (and changed his underwear presumably ) he must have IRC'd someone, who then posted the conversation in the forum:

Hacker said:

im sorry the thread i posted in the forums about my school website well the guys who owned the server they were alot clevere than we thought they mustve checked the referrrer and the looked at the thread and the cops have been called in and theyre checking up everyone who posted in that thread no dude im not jokingts anyone that posted in there is screwed i need a forum admin to lock it but im getting expelled and taken to court etc etc tommorow

[BliarOut]

judas said:

Update: we've caught one of the hackers! One of the sites on the hacked server is for a school and it was the site causing us the most problems. We checked the referrer logs and found several link-throughs from a hacking site. So we started reading the forums and some idiot who freely admitted being a pupil at the school was posting stuff about the site/server and other rather incriminating information. Not only that but he'd already e-mailed us over his 'concerns' about the server being slow! Long story short, left it a while to gather evidence while the school was away for half term, then gave the head teacher a call... Apparently he was dragged out of a lesson and told to come clean or face the very dire consequences. When he got home (and changed his underwear presumably ) he must have IRC'd someone, who then posted the conversation in the forum: Hacker said: im sorry the thread i posted in the forums about my school website well the guys who owned the server they were alot clevere than we thought they mustve checked the referrrer and the looked at the thread and the cops have been called in and theyre checking up everyone who posted in that thread no dude im not jokingts anyone that posted in there is screwed i need a forum admin to lock it but im getting expelled and taken to court etc etc tommorow

That'll teach him
Shame he didn't spend time on his spelling instead of playing with your server

[dilbert]

Well, good for catching him, but I fear you've just created a Hacking "god".

I've seen this happen before, and once word get's around you find their freinds seeking out their expert advice. Yada, yada, yada.

The less publicity the better in my view.

[judas]

'Hacker God'?



Nah - just another dumb script kiddie too full of himself to keep his mouth shut.

It's going to be interesting when we go over to 'interview' him later this week

[Pies]

Nice one

[guydw]

I would say that you don't need to spend $$$$$s on big firewalls (though you do need one, just not a big one - something like a PIX 501 is awesome and cheap) and you don't need to pay for very expensive consultants, you just need to implement a series of tried and tested changes to harden your OS and patch for known exploits. This is all stuff you can get off the web with a little effort.

Most hackers are very lazy and considerably less clever than they think they are, they just hit easy targets - much like most burglars and car thiefs !

However I don't agree with completely criminalising the little tts that hacked you (personally I think they should get a kicking then let that be the end of it..), the truth is that many security people have toyed with a little minor hacking in the past - doesn't make it OK, but they just need to be headed in the right direction !

[judas]

Well, our 'hacker' friend has just been suspended for two weeks and is facing permanent expulsion if he doesn't agree to cooperate with us. We have no interest in getting the police involved unless absolutely necessary, as this will only be a load more hassle. But if we have to we will, and we will absolutely crucify him - this idiot has cost us a lot of lost time and goodwill with our clients - and that translates into serious £££s. The school has done its bit and it's up to us what we want to do with him now - if we want to make an example of him then they won't stand in the way and this has been made clear to the kid and his parents.

Bubba may have a tender new playmate soon...