Anyone have experience of Cat 7 at 10G?

[Mr Pointy]

All the blurb says Cat7 is good for 100m at 10G, but does anyone have actual experience of installing or using it & found it doesn't make it that far in the real world?

[OldGermanHeaps]

84m run and a 78m run working fine for me at 10g to a roof mounted building to building microwave link. Exterior grade cat7 was very expensive though.

[rxe]

Do you really need 10G? As in, you have a use case that involves pushing 1GB/sec across a network consistently and reliably? If yes, I would use fibre - SFP modules are cheap enough, and fibre will be easier to install than copper. I’ve got a 60m fibre link to my garage for storage replication - easy to run the cable, been in there for 2 years now, seems to be bullet proof.

[Mr Pointy]

rxe said:

Do you really need 10G? As in, you have a use case that involves pushing 1GB/sec across a network consistently and reliably? If yes, I would use fibre - SFP modules are cheap enough, and fibre will be easier to install than copper. I’ve got a 60m fibre link to my garage for storage replication - easy to run the cable, been in there for 2 years now, seems to be bullet proof.

Yes. 60-odd NLE workstations are a lot easier to connect in copper than fibre, & to keep working in the future. There's zero chance the customer will bother with cleaning the LC connector ends if they ever move the workstations wheras they'll manage to plug an RJ45 back in with a good chance of success.

[rxe]

^ This.

Workstations can all be copper on a shared or dedicated switch. Either just use a switch uplink for the long haul or put in a dedicated mixed copper fibre switch as the bridge. My 10 G link is just the switch uplink ports.

[camel_landy]

+1...

Fibre for the uplink to a switch and then use regular copper for connections to the workstations.

M

[somouk]

+ another

Cat 6 locally to a switch and then fibre backbone to communicate between switches. Probably cheaper than buying Cat 7.

[Captain_Morgan]

Mr Pointy said:

Yes. 60-odd NLE workstations are a lot easier to connect in copper than fibre, & to keep working in the future. There's zero chance the customer will bother with cleaning the LC connector ends if they ever move the workstations wheras they'll manage to plug an RJ45 back in with a good chance of success.

It’s an interesting requirement with 60 + NLE’s what does the back end storage look like?

How many 10G connections are used to keep up with that volume of workstations?

I’m guessing that the number of up-links to the file servers would be significant if you used local copper cabling for the workstations is 100G interlinks a viable option?

[camel_landy]

Captain_Morgan said:

It’s an interesting requirement with 60 + NLE’s what does the back end storage look like?

Ceph?

M

[Captain_Morgan]

camel_landy said:

Ceph?

M

I was thinking more from the number of file servers, there storage and network connectivity

But I’ll have a read about Ceph

[xeny]

Mr Pointy said:

All the blurb says Cat7 is good for 100m at 10G, but does anyone have actual experience of installing or using it & found it doesn't make it that far in the real world?

You'll be fine.

I've got an approximately 95m run made up of of patch cable, structured Cat5e, patch cable, structured Cat 6a, patch cable which has a 10G LAG across it.

Given the standard doesn't allow patch cables in the middle of runs, or 5e cable, there must be a lot of headroom in a remotely to spec installation

[camel_landy]

Captain_Morgan said:

camel_landy said:

Ceph?

M

I was thinking more from the number of file servers, there storage and network connectivity

But I’ll have a read about Ceph

Annoyingly, most articles about Ceph lose readers with techno-babble, before they've managed to get their head around some of the basics of what it is and what it allows you to do.

My take on it is that Ceph lets you cluster lots of storage servers in a way which redundant, resilient, scalable, configurable, etc, etc... It then gives you very high throughput, as it's reading & writing to the media in parallel. In turn, this allows you to use 'cheap' HDD as the storage media, to keep costs more reasonable but the way they're clustered then gives you the performance.

http://www.45drives.com/blog/uncategorized/what-is...
https://www.fasthosts.co.uk/blog/what-ceph/

IIRC - Ceph is being used by the likes of CERN, media streaming companies, Cloud providers, etc. But you can even use it across a small handful of servers, in a more modest environment.

https://ceph.com/en/

HTH

M

PS Apologies for heading slightly OT.

[xeny]

I've only used ceph for a project, I've not had to admin/implement it.

I found throughput was good, but metadata performance (even obtaining directory listings) was very sluggish.

I've no idea though if that was down to poor implementation or the nature of the filesystem.

[camel_landy]

xeny said:

I've only used ceph for a project, I've not had to admin/implement it.

I found throughput was good, but metadata performance (even obtaining directory listings) was very sluggish.

I've no idea though if that was down to poor implementation or the nature of the filesystem.

I'd suggest that would be more down to the implementation and the way the environment was structured. Dropping things like metadata, onto fast access devices (eg NVMe), makes a big difference.

Once you start getting drawn in, there's loads you can do to tune it to your environment.

M

[xeny]

Seems likely. This was before NVMe was trivially available, but I remember that at the time a simple ZFS or XFS FS with the same data on it (a few hundred TB from memory) was far more pleasant to deal with.