PC Advice Multiple Cores

[stevieb]

Hi,

I have a techy query about Win XP and the number of processors it can support.

I have been give a few grand from work to spend on a dedicated number crunching machine, but it should double up as a good games machine aswell..

Current Spec are Dual Intel Xeon Quad Core 2GHz Processors with 8mb onboard Cache. 8Gig Ram and 800 Gig SAS HD. and a decent Graphics card..

So how does XP address the 8 processor cores and allocate jobs to each one.

I have some software that allows upto to 10 instances to run at a single time so what i am hoping is to run 8 instances of the program and XP to allocate the 1 core and 1gig ram to each instance OR am i hoping for too much!

Thanks

Steve

[roadsweeper]

I can't answer your question in detail I'm afraid, but doubt XP will allocate cores and RAM so precisely as you hope - someone by all means correct me if I'm wrong. As I understand it, you'd need to use some pretty clever (and quite expensive) virtualisation software, such as VMWare ESX Server 3, to achieve that. However, that's not to say it won't run well...

[roadsweeper]

One other thing to look out for is to ensure you get the fastest memory supported by your system as many off-the-shelf systems come with slower RAM as it's cheaper.

[stevieb]

roadsweeper said:

I can't answer your question in detail I'm afraid, but doubt XP will allocate cores and RAM so precisely as you hope - someone by all means correct me if I'm wrong. As I understand it, you'd need to use some pretty clever (and quite expensive) virtualisation software, such as VMWare ESX Server 3, to achieve that. However, that's not to say it won't run well...

I know i am hoping for a lot...

I currently have a dual core PC at the moment and when i run 1 instance of the software it allocates one core to 100% usage, and the other is idle. then i can start a second instance and this is then allocated to the second core 95% for the software and 5% for XP.

I dont know how it allocates the memory but i have yet to go above 1gig of memory allocated to each process, hence why i am speccing this up to have 8cores and 8x1gig memory chips.

Moeny is not really an issue as the machine is costing approx £5k I just need something more powerful.

run time at the moment are into the 250-300 hours, for a complete package running on a single core machine 150-160 hours on a dual core. so i am hoping to bring this down to around 50 hours with a dual quad core machine.

[tank slapper]

It is possible to allocate tasks to an individual processor by setting the processor affinity. This is found in task manager by right clicking on the individual process you want to set. If you need to change the affinity permananently you can use this method. That changes the executable file, so make sure you have a backup of it.

[BliarOut]

Have a nosey on the MS support site... Prior versions were only licensed for four CPU's!

[stevieb]

BliarOut said:

Have a nosey on the MS support site... Prior versions were only licensed for four CPU's!

Already sorted that one licensed for 2 physical CPU and upto 32 Logical Cores.

[stevieb]

tank slapper said:

It is possible to allocate tasks to an individual processor by setting the processor affinity. This is found in task manager by right clicking on the individual process you want to set. If you need to change the affinity permananently you can use this method. That changes the executable file, so make sure you have a backup of it.

Thanks for that so if i use that, i would have to create 8 copies of the program and allocate each one to a individual core?

If so can i test this on a dual core machine by switch a task from Core 1 to Core 2 and see of the program switchs in the task manager to demostrate to managment that this method works
Thanks

[PJ S]

Maybe you should discuss this further with Armari since this is their area of expertise.
Equally, a Mac Pro would do the same job, especially with bootcamp if the software is Windows only and not available for Mac/'Nix.

Easily done with Armari for £3150 with a single 8800 GTX and a few other removals and upgrades (4GB 8500 RAM, FW800 card, 750 GB drive, Quad Xeon)
Mac Pro would be similar sort of money with ATI 1950.

[tank slapper]

stevieb said:

Thanks for that so if i use that, i would have to create 8 copies of the program and allocate each one to a individual core?

If so can i test this on a dual core machine by switch a task from Core 1 to Core 2 and see of the program switchs in the task manager to demostrate to managment that this method works
Thanks

I think that you would need separate copies for each instance you are intending to run if you intend to make the setting permanent. Before you go down that road though, it would probably be as well to test the performance without worrying about affinity, as Windows isn't too bad at spreading the load around. You may find that any benefit is marginal, but I suspect it will depend on what type of job it is. Also, if you are talking about jobs that take many hours to compute and don't need it automated, it is probably easier to set affinity manually. That way you won't need multiple copies.

The other thing to bear in mind is memory limitations. Windows XP Pro can only handle 4Gb of physical memory - see here. If you want to go above that, you will need the x64 version which can handle 128Gb, or one of the server operating systems that support PAE see here and here.

It is probably a good idea to speak to someone who is used to dealing with systems with lots of RAM as some ways are better than others, depending on what you are doing.

[ThatPhilBrettGuy]

hhhmmm, that you say each prog wants about 1GB RAM makes me think that there quite a bit of memory reading going on. This will be the bottleneck. You'll probably get no more performance than 2 twin core processors... Being a desktop machine, it won't have NUMA memory architecture (google is your friend).

An example, we ran a 4 proc, twin core Intel machine with 8GB of RAM with one of our high memory load apps, and got 29,000 transactions a second. A 4 proc, twin core AMD with much lower clock speed procs, but with NUMA got 81,000.

[SneakyNeil]

Personally I've still got a little alarm bell in my head re. 8 processors in XP. A quad-core CPU has four physical CPU's on one core, not virtual ones. Virtual CPU's = hyperthreading, physical cpu's = multicore. How this applies to XP SP2 (which was released long before quad core CPU's came out) I don't know but I think it warrants further investigation.

eta, some benchmarks from various dual xeon configs here -
http://tomshardware.co.uk/2006/12/06/

They used XP64 so looks like you might be OK.

Edited by SneakyNeil on Thursday 8th March 09:25

[ATG]

Same concern as ThatPhilBrettGuy. Sounds like your software wasn't to take any particular advantage of running on a multi-core PC, so have you considered getting several cheaper machines, networking them and running your multiple processes across multiple machines? If your process requires substantial memory and disc access, adding extra processors to the same machine will give rapidly diminishing returns. Several cheap boxes might give you more clout, plus the benefit of redundancy.

[stevieb]

ATG said:

Same concern as ThatPhilBrettGuy. Sounds like your software wasn't to take any particular advantage of running on a multi-core PC, so have you considered getting several cheaper machines, networking them and running your multiple processes across multiple machines? If your process requires substantial memory and disc access, adding extra processors to the same machine will give rapidly diminishing returns. Several cheap boxes might give you more clout, plus the benefit of redundancy.

The problem lack of space for machines. plus a project in future is to re write the software to take advantage of multiple cores more effectively.

[tank slapper]

SneakyNeil said:

Personally I've still got a little alarm bell in my head re. 8 processors in XP. A quad-core CPU has four physical CPU's on one core, not virtual ones. Virtual CPU's = hyperthreading, physical cpu's = multicore. How this applies to XP SP2 (which was released long before quad core CPU's came out) I don't know but I think it warrants further investigation.

That isn't a problem. Microsoft consider a multi-core processor to be a single physical processor regardless of the number of cores, so his set up would have 2 physical processors, even though there are 8 actual cores. Clarification from Microsoft

[ATG]

stevieb said:

ATG said:

Same concern as ThatPhilBrettGuy. Sounds like your software wasn't to take any particular advantage of running on a multi-core PC, so have you considered getting several cheaper machines, networking them and running your multiple processes across multiple machines? If your process requires substantial memory and disc access, adding extra processors to the same machine will give rapidly diminishing returns. Several cheap boxes might give you more clout, plus the benefit of redundancy.

The problem lack of space for machines. plus a project in future is to re write the software to take advantage of multiple cores more effectively.

Ah, fair enough. I'm trying to come up with an excuse to justify building a cluster based on networked mini-itx mother boards.