Network Performance Testing

Hope everyone is having a good holiday season.

I wanted to post more of a question than anything.  At my work, and in several consulting related situations, I interact with Spirent equipment for various testing needs.  In the last year, it has become progressively more difficult to get responses and information out of Spirent for a number of reasons.  This has ranged from support calls, to software fixes and questions, to sales inquiries.

Are others seeing this?

Spirent has a lot of equipment and customers, and I have been contacted by a number of existing Spirent customers with similar concerns over the last year.  Most people do not have the demands and requirements (at least those I talk to) that my company does for performance test vendors, and as such, haven’t run into any of the problems we’re experiencing, or at least not to the degree we are.  Another way of saying this – we feel the pain more than most, faster than most.

I wanted to see if others are experiencing similar issues, or if there are things I am not aware of that should factor into this opinion.

Shenick announced today that they are partnering with Fanfare to add full plug in support to the Shenick diversifEye™ product.  According to the press release, this will allow users of Fanfare’s iTest product to control and manipulate per-flow testing from the Shenick product.

A lot of the press release is focused on Service Providers and wireless markets, which is an area that Shenick focuses on more than many of the other players.  Direct from the press release:

“NGN and LTE services and applications bring with them increased quality of service (QoS) and quality of experience (QoE) demands. Per-flow test and measurement is the only mechanism to guarantee performance at these granular levels enabling providers to pinpoint application and individual user issues accurately”

I think it’s great that this support has been added on both ends so that users of iTest can take advantage of existing or new Shenick equipment.  I hope there will be more details about how deep the integration goes in the coming weeks.

One interesting thing is the fact that both Fanfare and Shenick are part of the TesLA Alliance – I wonder how much the proposed TesLA standards have influenced this solution, and if we’ll see more of this in the coming months since the standards are getting closer to being approved…

Spirent announced a few weeks ago a new solution, TestCenter Virtual, for validating the performance of all elements of the data center and cloud computing environments including virtual machines (VM), servers and storage devices.

In examining the details, it appears that TestCenter Virtual is a module that is installed in each VM which will allow for more granular control of the overall test by actually generating traffic from the VM’s themselves.

The product has been selected as a finalist for Best of Interop 2009.

I think it’s a great idea, along the lines of IxVM announced earlier this month by IXIA.  Allowing purpose-built performance test equipment to control virtual machines will allow for folks to validate all of the virtual infrastructure, and possibly also be a great way to scale testing without having to purchase more performance equipment.

I am very interested to see these products once they are out and available to the general public.

IXIA announced today their plans to perform a big test at Interop with Force10 for 10GE switching.  It will require 280 IxYukon 10GE test ports, and end up sending and receiving 5.6 terabits per second of traffic.  Quite a bit of traffic!

Seems that  lot of people are doing the bigger/faster/more test stuff – Spirent did a 10GB test back in October that was, at the time, the largest.  BreakingPoint also did a big test with Juniper back in March, although that test was markedly different in the goals and type of test.

Those of you going to Interop (I’m not) should stop by and let us know what it looks like.

IXIA announced a couple of days ago a new testing solution for virtualized data centers – IxVM.  There aren’t many details on the product yet other than the press release, but it seems pretty straightforward.

From the press release, it seems to be a version of IxChariot that will allow you to aggregate reporting and management of many endpoints distributed on VMware virtual machines.  This would allow you to test virtualized environments better than most other solutions I’ve seen so far.

Again, there’s not much info that I can find about the details – it will be interesting to see how integrated this tool is with VMware and virtualized environments in general.  It will also be interesting to see if support for other virtualization technologies is planned.

Announced yesterday, Gale Technologies has joined the TesLA Alliance.  I’m happy to see more folks joining the alliance, and I look forward to working with Gale in my capacity as a customer advisory board member.

Gale’s technology is pretty neat – something I’m investigating at work as I mentioned in a previous post.   I’m particularly interested in a system that will automatically provision and configure test architectures, and allow for dynamic scheduling and interaction with those test architectures from various different automation systems.

Our system now is home grown, and while it serves it’s purpose well, doesn’t allow for as much flexibility as many of the commercial products.  We have yet to decide whether we’ll buy one, or just add on to our own – still need to finish our evaluation of the various solutions out there first.

More on that later…

Interesting article and video on BreakingPoint’s site about a recent test of a Juniper Networks SRX5800 @ 150 Gbps – 109Gbps of blended application traffic and live security attacks, 30Gbps of intrusion prevention.  Quite a lot of traffic all at one time.

I’m sure this was possible as part of BreakingPoint’s other breakthrough back in February on inventing a new architecture to deliver 40Gbps of blended traffic from one device.

The most interesting thing to me is the blended application traffic approach – being able to use their library of application protocols in a blend for a more realistic test scenario.  I know many of the other vendors in this space have good application support, but the act of configuring a more realistic traffic mix always seems to be way too hard to configure, and usually beyond a test engineers patience threshold.

Given where this industry is today, more wizard-ized or componentized solutions are necessary to allow customers to get the most out of their test equipment.  Providing all of the bells and whistles, the dials to change everything, used to be the main thing I remember asking for from test vendors.  This was when you needed the extra SSL cipher support, or the option in the HTTP header to do something.  Don’t get me wrong, this is still a requirement, it’s just expected now that when a vendor comes out with protocol support, that it’s supported from end to end, just like a client would have.

But now, we need easier ways to configure a more realistic traffic profile, network profile, WAN profile, etc.  Being able to sniff stuff off the wire and create a traffic mix is a great way to approach this, but not all that easy to do.  The next best thing is providing easier ways to configure mixes of traffic all at one time.  It sounds like BreakingPoint is on the right track here, and to boot, it’s really fast!

Shunra announced recently that they’ve integrated with HP Performance Center 9.5.  The product is called Shunra VE Desktop for HP Software, and allows each load generator in the HP Performance Center to be setup to emulate different WAN conditions.

From the press release:

Some typical uses of Shunra VE Desktop for HP Software include the detection of:

Latency impact – Latency causes the transactions and sessions to stay open longer than they would on the LAN. Without incorporating the WAN effects of delay into a load test the memory usage, thread usage, and other critical server resources can be significantly understated.

Bandwidth limitations – Bandwidth constraints affect transactions “in-flight,” causing them to take longer to complete their transmission and ultimately impacts important server resources.

Network impact – Where specific response time goals are being targeted, it is important to incorporate the effect of the network into existing testing processes to examine the impact on total response time. Shunra enables you to accurately assess total response time for remote users prior to deployment and can analyze the breakdown of total response time into the client, network, and server components. This assists in diagnosing and resolving transactions that will not consistently meet response time goals.

Looks like a pretty cool add-on for those of you out there with Performance Center.  I think testing with more real-world conditions like this is not just a nice-to-have anymore, but a requirement in today’s test environments.

I’m on the Customer Advisory Board for the TesLA Alliance and we had our second quarterly meeting this last week.  I missed the first meeting due to being sick, so it was great to finally meet the rest of the TesLA Members and the Customer Advisory Board as well.

I am really excited about all of the new things that we’re planning to do, and very appreciative to be involved with the entire alliance.  The caliber of all of the members and board is top-notch, and there are many very exciting dicsussions going on that will forever change this industry in the positive.

Why is it so exciting to me?

Well, first you have a collection of Members that span the performance, functional, and lab automation arenas.  Some of these vendors are bitter rivals, yet they met together, and all agreed on many things that will help to not only unify the market, but provide far more benefits to end customers.  There are even more vendors that are interested, and attended the first part of the meeting to see what it was all about.

Second, you have the goals of the alliance, which are exactly what I’ve been looking for.  Defined and published standards.  Framework to allow vendors to communicate between each other.  Completely open to all vendors.  An alliance focused on all of these things, and not on building their own product.

Third, the momentum behind this is amazing, and the amount of progress that has been made in the last few months shows that this is being done at the right time, and the right place.  So many people agree that the industry needs this ASAP.

Finally, the customer advisory board – my colleagues are all top-drawer folks from varying parts of the industry.  One of the members was adding up the experience we all have together and we came up with something around 80 years!  We’re the reality check for the rest of the alliance to make sure that what they’re coming up with meets customer expectations.  It’s a privilege to work with these folks on such a critical standard for our industry.

What’s next?

I would like to see even more vendors participate in the alliance.  There are some noticeable key players that are still playing politics, when they should really just put those aside and get on board.

There’s a lot more planned, and a lot of new things happening every week.  Check out the Activities Calendar, or subscribe to the RSS feed for the site.  I’m looking forward to the ongoing participation, as well as the new folks who will be joining.

Does anyone know what’s happened with Gale Technologies?  Their news page has been empty for months, and the only other updates are a few customer stories…

Many of you will remember that Gale was formed by the merger of QuikCycle and Edentree which I wrote about back in October.  Both were pretty major players in lab and test automation.   I sure hope we’ll see something new from them soon!

UPDATE: They are very much still alive – I bumped into some of them a few days ago at the TesLA Alliance Quarterly Meeting.