Could seomone help out with specs for a server
Discussion
I'm way out of touch with hardware and OSs these days and requirements for a LAMP installation.
I need to spec a server to run Apache, MySQL & PHP. O.S. to be Linux - probably RedHat as I'm more used to that.
Scope is as follows.
The server is posted events by (currently) for Network Monitoring Clients. Each time a Client sees a certain packet on the Network it sends an event to the server. The event information is sent via an HTTP Post message.
All the events at the server are stored in a single table. The table comprises a few colums that hold the date-time, protocl, service and envet type information. Data fields that describe the event are inserted in upto 60 separate colums. The colum to insert these data fields are determined on the fly by looking up other tables to determine the correct column tto use. These columns are varchar(255). So we can store upto 60 fields of information per event and each field can have a maximium of 255 characters in length.
The tables that describe the colums to use during an insert can be considered as static. The only table that is going to get any major level of inserts is the event table. This is heavily indexed to speed up searches.
Our field definitions table is likely to run to about 35,000 records, each record only really being few 100s of bytes long.
The event table will grow at a max rate of 50-100 records per second.
Currently the processing that does event inserts is running in PHP, which parses the incoming insert request, determines which information needs to go into which field and then creates an SQL insert statement which is then called on the DB. So if the incoming insert request contains 10 fields of information there are typically 10 lookups into the DB to determine which column in the event table that information should be put into.
I don't know if it would be better to move all this activity into a stored procedure in the DB yet, or if it is ok to leave PHP doing the work. (ideas?)
Next we come to the clients that connect to the DB, probably around 10. Clients opperate in either real-time or historic mode.
In real-time, AJAX is used to pole the database every couple of seconds (configurable) to ask for the most current list of events. In historic mode the client askes for events matching a set of criteria, comprising a date-time, event type, event field matching 'xxx' etc.
Finally, we are also tracking state on the events that are posted, so there will be inserts into other tables that track these events based upon the fields supplied, so upon receiving an event insert request there could well be some extra scans of the events table in order to match up for example requests and responses.
Hardware:
My gut feeling is that a couple of Dual-Core Xenon processors with 4GB RAM should easily handle this, but I can't put 'gut-feeling' onto the hardware proposal.
Any one got any insight, links, rules of thumb for speccing out hardware to address these load levels?
Many thanks
Ex
I need to spec a server to run Apache, MySQL & PHP. O.S. to be Linux - probably RedHat as I'm more used to that.
Scope is as follows.
The server is posted events by (currently) for Network Monitoring Clients. Each time a Client sees a certain packet on the Network it sends an event to the server. The event information is sent via an HTTP Post message.
All the events at the server are stored in a single table. The table comprises a few colums that hold the date-time, protocl, service and envet type information. Data fields that describe the event are inserted in upto 60 separate colums. The colum to insert these data fields are determined on the fly by looking up other tables to determine the correct column tto use. These columns are varchar(255). So we can store upto 60 fields of information per event and each field can have a maximium of 255 characters in length.
The tables that describe the colums to use during an insert can be considered as static. The only table that is going to get any major level of inserts is the event table. This is heavily indexed to speed up searches.
Our field definitions table is likely to run to about 35,000 records, each record only really being few 100s of bytes long.
The event table will grow at a max rate of 50-100 records per second.
Currently the processing that does event inserts is running in PHP, which parses the incoming insert request, determines which information needs to go into which field and then creates an SQL insert statement which is then called on the DB. So if the incoming insert request contains 10 fields of information there are typically 10 lookups into the DB to determine which column in the event table that information should be put into.
I don't know if it would be better to move all this activity into a stored procedure in the DB yet, or if it is ok to leave PHP doing the work. (ideas?)
Next we come to the clients that connect to the DB, probably around 10. Clients opperate in either real-time or historic mode.
In real-time, AJAX is used to pole the database every couple of seconds (configurable) to ask for the most current list of events. In historic mode the client askes for events matching a set of criteria, comprising a date-time, event type, event field matching 'xxx' etc.
Finally, we are also tracking state on the events that are posted, so there will be inserts into other tables that track these events based upon the fields supplied, so upon receiving an event insert request there could well be some extra scans of the events table in order to match up for example requests and responses.
Hardware:
My gut feeling is that a couple of Dual-Core Xenon processors with 4GB RAM should easily handle this, but I can't put 'gut-feeling' onto the hardware proposal.
Any one got any insight, links, rules of thumb for speccing out hardware to address these load levels?
Many thanks
Ex
It's one of those times when you simply need to spec to the best of your budget.
I'd say a RAID array, ideally 10 to give as many spindles as possible (for maximal disk I/O and Data redundancy, with no performance hit (raid5) with lots of RAM (DB's love to soak up the RAM), cpu will be less critical if all the above is taken care of, but with Dual QuadCores being so cheap these days, it'd be silly not to get the best you can
J
I'd say a RAID array, ideally 10 to give as many spindles as possible (for maximal disk I/O and Data redundancy, with no performance hit (raid5) with lots of RAM (DB's love to soak up the RAM), cpu will be less critical if all the above is taken care of, but with Dual QuadCores being so cheap these days, it'd be silly not to get the best you can
J
JamieBeeston said:
It's one of those times when you simply need to spec to the best of your budget.
I'd say a RAID array, ideally 10 to give as many spindles as possible (for maximal disk I/O and Data redundancy, with no performance hit (raid5) with lots of RAM (DB's love to soak up the RAM), cpu will be less critical if all the above is taken care of, but with Dual QuadCores being so cheap these days, it'd be silly not to get the best you can
J
Hi Jamie,I'd say a RAID array, ideally 10 to give as many spindles as possible (for maximal disk I/O and Data redundancy, with no performance hit (raid5) with lots of RAM (DB's love to soak up the RAM), cpu will be less critical if all the above is taken care of, but with Dual QuadCores being so cheap these days, it'd be silly not to get the best you can
J
hmmm.... budget has dropped dramitically since we last spoke. looking at around £3,500, I want to put 10-15% on top of that for our handling, installing the OS etc.
I'm just curious as to whether that hardware will handle the requirements. My 'gut feeling' is that with this budget we will over spec. This is not going to be a 24/7/365 system, so even though RAID 10 would be good, if we spec RAID 1 and just mirror a single system disk we are covering our arses in the event of a disk failure.
2 dual core CPUs will surely fly this application? Is 4GB RAM enough - should we look at 8?
It's very much a case of giving the customer something that will work well, making us a few folding beer tokens on the side and ensuring that 3 weeks after the installation we don't have to approach the customer and say we need more money for RAM (or what ever)...
I just have no experience in speccing such systems.....
So, do I just go for the cheapest bare bones Dell Poweredge 1950 and stick a bit more RAM and another CPU into it?
best
Ex
Gassing Station | Computers, Gadgets & Stuff | Top of Page | What's New | My Stuff