Sunday, April 27, 2008

$120,000 of Cisco switching, but cant deal with iSCSI and numbers dont add up

A few years back we order two Cisco 4510 to replace our two Cisco 4006. At the time I am thinking about moving to iSCSI away from Fiber Channel.

So you can select diffrent Engines with this product and when I have this types of options I buy the BIGGEST. and I was thinking I was getting the best (Its Cisco its the best, its cost $120,000).

So I had my self Cisco Catalyst 4500 Series Supervisor Engine V-10GE with 8 x 48 Port 1000mbits PoE line cards.

2/3 years later I find out..

a) the line cards have 48 x 1000mbits dont support Jumbo Frames (up to 9216 bytes) when I was thinking that Jumbo Frames was a Engine feature.

b)
Per slot switching capacity is only 6 Gbps when the total for switch 136 Gbps. I have 10 slots... so why dont I have a min of 13.6 Gbps per slot? If I have a 48 Port line card I should be able to pump thu 48 Gbps.

c) Every 6 ports on 48 Port line card shares 1 Gbps. So I can only get 8 Gbps, but does not matter as the slot only supports 6 Gbps.

So I got 24u (1/2 a rack) of tin.. which cost $120,000.. and does nothing for iSCSI. I really think I have to review my want to buy Cisco. I think Cisco is amazing company but I really hoping for a lot more.

Plus this not the only product. If you played with Cisco Router 3640 it can support up to 8 E1 interfaces, but it can deal with more then 2 1/2 with H.323.

1 comment:

Raman, RNDTS said...

Hello,

Before venturing into expensive switches like Cisco, give other switches in the market - never go just by the name. Jumbo frames and flow control are very important in any iSCSI solution.

With any iSCSI solution in market does not support more than 3 or 4 GbE ports. In general, the bottleneck is always on the GbE ports no matter what drives you use - SAS or SATA. If you start start stacking up boxes with few iSCSI ports then managing the switch and ports alone becomes a challenge.

Never spend on expensive components like Switches, GbE NICs etc, but carefully consider the future transition as well when designing iSCSI solutions.

The reason iSCSI is chosen to reduce costs and at the same time not compromising on the performance.

Go with modular component system when designing storage solutions because technology keeps changing in a short span.

As you know 10GbE is the future (or at least for next 5 years) and will probably replace FC in data centers or new FC investments will stalled.

How will your storage be affected when 10GbE becomes a standard? How am I going to spend to move to 10GbE? How am I going to justify the cost? These are some of the questions one should ask before moving to iSCSI.

There is ONLY one product today in the market that supports a modular upgrade to the iSCSI storage system without changing a thing other than the IP Switch. All servers will still be able to communicate with iSCSI storage system but you will have a better iSCSI solution in place for a fraction of the cost one would spend in changing the infrastructure/storage.

There is a product from iStor that is available with multiple GbE ports (8 to be precise) as well as 10GbE with single and dual-controller options. Say today you buy a 8 port version to meet your requirements, moving forward you could upgrade just the controller to 10GbE controller by just unplugging the 8x1GbE controller and plugging the new 10GbE without changing any storage configuration or settings and the system will just work fine, but only with better infrastructure.

If you compare the cost of a 10GbE switch today, for example, from Fujitsu, it costs less than 10K retail with 12x10GbE ports.

One has to carefully consider the solution they are investing in - hardware or a software solution? Any software solution especially for RAID and Storage Virtualization is going to be slower so give it a careful consideration before investing in iSCSI - DO NOT GO BY BRAND NAMES.

With just 8x1GbE or 1xGbE (with single controller) one could achieve 800MB+ and 1102MB+ respectively and scalability upto 36TB today with SATA and ~16TB with SAS - the controller supports both SATA and SAS.

Hope this helps,
Raman
RNDTS