Research supports a specific theory depending on the amount of funds allocated to it.
Unknown
Let’s say that you need to buy a wifi router for your home. Nowadays, we’d do a Google search for wifi routers, scout on amazon or flipkart/ walmart or your preferred e-commerce shopping site for the router with the best features and a price that suits your budget. One important criterion in our search would be a comparison of the features between devices from different vendors. What we’d love to have is a validation of sorts from someone about the authenticity of the claims from the vendors. Which is where independent test reports from impartial sources would be helpful.
What are they?
Third-party test report providers are just what the name suggests. They are companies or entities that test network equipment against their definitions of “real-world” scenarios. Think of real-estate lawyers who whet property documents and validate that the property is genuine. There is this one lawyer who applies the same check-list against all the property documents presented to him of different lands across various geographies. Apply this corollary to the networking world, replace lawyer with say, EANTC, NSSLabs, Miercom, etc. and the land with networking equipment and you get the picture.
OK, so why do we need them?
Vendors market their equipment based on certain test parameters which are carefully engineered to provide the best possible performance of the device. Nothing wrong in that – the world does revolve around selling. Now, if each vendor applies their own twist to the way the numbers are presented, how is a customer to decide which of the devices meet their requirements? Here step in these independent entities that apply the same test procedures and parameters across different equipment. This should lead to highly comparable and authentic test results that show the devices in their true colors, so to speak.
And the problem is…?
Who polices the policemen? Where there is any entity providing reports, there is scope for influencing the results in anyone’s favor. When Governments can be influenced, the media can be influenced, why can’t these “independent” test report providers? The tests, when conducted in partnership with the device vendors can provide vastly different results as opposed to a truly independent test. Of course, the routers, switches, security devices are incredibly complex pieces of equipment and increasingly, can be engineered to provide either more performance or more features. This flexibility is important for their customers, but exposes the risk of adverse test reports when conducted without the vendors’ support. Which brings us to a new problem – how do we ensure a truly authentic and independent report that customers can trust, when the tests need the presence of the vendor? So, the third-part testers indulge the vendors. When there is indulgence, there is always a scope of influence and we are back to the first question – who polices the policemen.
Hmm. Solutions?
This is an incredibly difficult problem to solve. Not privy to economic theories, I am sure there must be one for this kind of a problem. Ultimately, I think the maxim of research supporting… funds allocated to it can easily be replaced with “Test reports supporting a particular vendor is directly proportional to the amount of time and energy spent with them”. Time and energy is not free, they come at a price.
Seeing is believing. Until customers see the product performing for themselves, in their simulated or actual networks, it is not worthwhile to place too much emphasis on independent tests. If this is not possible, at least go by other customer reviews. I know that I rely on them when shopping online, now why wouldn’t customers of (far more expensive) networking equipment?
Mithun's Musings 