Network Testing Labs: 24x7 Application Monitoring

We discovered Heroix's eQ Management Suite is the most capable, most scalable and - by far - most flexible tool for monitoring servers and applications.

Imagine a diligent, busy network administrator continuously running from server to server, checking each machine's health, network connection, operating system and applications. Now imagine the network administrator doing these chores 24x7, unblinkingly and without taking a break. Indefatigable, conscientious and above all quick, the person checks scores of machines each minute. That employee would certainly deserve a huge raise.

Automating a network administrator's watchful eye isn't easy. Nonetheless, a small number of software vendors claim to have done just that. Professing to make a network administrator more productive and efficient, these firms offer network monitoring tools that closely observe the operations of applications and servers, then alert the administrator if the tool finds a problem.

A perfect application monitoring tool rapidly checks every conceivable operational parameter associated with your applications, your servers and your operating systems. It can detect simple threshold violations as well as more complex violations involving sustained error activity for given amounts of time at particular times of the day. For each specific situation, a perfect tool can notify administrators via e-mail, pager call and SNMP alert. The tool can also, when appropriate, automatically fix the problem by restarting a failed software service, restarting an application or rebooting a server. The user interface is intuitive and responsive, and an administrator can operate the tool locally through a native operating system management console as well as remotely through a Web browser. The perfect tool has excellent, comprehensive reporting capabilities, it scales well, it's robust and reliable, it's easy to install and it comes with clear, easy-to-follow documentation.

So which application and server monitoring tool is best for your company? To determine the best application and network monitoring tool, we tested three products from vendors claiming to offer enterprise-ready automation of a sizable number of a network administrator chores. Our review put Heroix's eQ Management Suite 2.0, BMC Software, Inc.'s PATROL for Microsoft Windows Servers 3.0 and PATROL for Unix 9.0 and NetIQ Corporation's AppManager 5.0.1 through their paces. See "Network Testing Labs' Testbed and Methodology" for a description of the tests.

Heroix's eQ Management Suite emerged the clear winner in our evaluation, excelling in flexibility, platform support, scalability and reliability. We found that it can detect, warn, report on and sometimes even fix server and application problems on even the largest network. eQ Management Suite earns Network Testing Labs' World Class award for best application and network monitoring tool. (see Scorecard)

We found eQ Management Suite to be much more than just an application monitoring tool. Its reports can form the basis for serious capacity planning, its alerts can be an important part of the data you collect in a network management system (such as Hewlett-Packard OpenView) and it can significantly increase the uptime and availability of your servers and key applications.

Monitoring

eQ Management Suite impressed us with its range of supported applications, operating systems and servers. Most impressive was its thorough, detailed support for individual applications and operating systems. For these, we found that eQ monitors every possible parameter and behavior.

Heroix terms each monitoring component a Solution. Each Solution is a small, well-crafted agent that eQ automatically installs and activates when it detects the operating system or application that specific Solution is for. eQ contains 49 Solutions for various applications, from Active Directory Services to Windows Load Balancing Service (WLBS), from Compaq Insight Manager and Dell OpenManage to Microsoft Windows Server, from Oracle, DB2 and SQL Server to the BEA Systems WebLogic application server environment, from Apache to Internet Information Server (IIS) and from Lotus Notes and Domino to Exchange. Other Solutions monitor, for example, Service Level Agreements (SLAs) that you've established in your company's computing environment. In Heroix parlance, sensors within each Solution continuously test the individual parameters that the Solution monitors for.

eQ's agents can monitor nine operating systems - various flavors of Windows Server, Solaris, HP UX, Compaq Tru-64, AIX, Red Hat Linux, OpenVMS and Novell NetWare. eQ also contains specific support for Cisco devices, which eQ activates on any machine running SNMP. Any eQ-agented server running SNMP can run eQ's Cisco Solution to monitor Cisco devices. Heroix eQ agents and the management console use TCP/IP to coordinate and exchange event data. eQ stores server, application, operating system, event and performance data in a SQL Server database.

NetIQ AppManager, on the other hand, has 48 application-specific monitoring components plus seven ResponseTime components designed to track the performance of certain software products. AppManager supports Windows NT, Windows 2000, HP-UX, AIX, Red Hat Linux, Solaris, and Novell NetWare. However, AppManager lacks support for OpenVMS and Compaq Tru-64. AppManager also lacks support for the DB2 and Sybase relational database products. It does have some support for such products as Brocade switches, WebSphere, ATG Dynamo, Netfinity servers and Siemens servers.

In contrast, the much more expensive PATROL has over 80 separately licensed modules for customers to select from. BMC's plethora of module offerings is comprehensive but somewhat confusing. Picking and choosing the right PATROL modules can be an arduous task.

Like eQ Management Suite, PATROL supports Windows, Linux, NetWare and OpenVMS. PATROL supports some flavors of Unix plus the AS/400 and OS/390 platforms. Unlike eQ and AppManager, PATROL also offers support for SAP and Informix.

BMC terms PATROL's separate performance tracking modules Perform & Predict components. These components work with PATROL's application monitoring modules in a manner similar to the way eQ Management Suite's and AppManager's performance tracking modules keep an eye on resource utilization. NetIQ calls its performance monitoring components AppManager ResponseTime modules.

The number of applications a monitoring tool supports is less important than the quality i.e., depth) of each application monitoring effort. Moreover, the flexibility with which a customer can tailor an existing monitoring component to that customer's particular computing environment - or, better yet, create new monitoring components - is crucial. We found that, in general, eQ Management Suite examines more application parameters than either PATROL or AppManager. For example, eQ has 36 individual sensors for Microsoft SQL Server, while PATROL monitors 14 SQL Server parameters and AppManager monitors 16.

eQ Management Suite also offered us an advanced, more complex set of thresholds for its detection of problems. For appropriate Solutions, for example, we could specify which days of the week or days of the month as well as time intervals during each day that a Solution should look for resource consumption that was either too high or too low.

We absolutely loved eQ Solution Studio. A visual design environment for creating new or altering existing Solutions, the Studio is a godsend to companies that use applications in ways that the application's vendor or even the monitoring tool vendor never imagined. A bank, for example, could fairly easily "instrument" eQ to monitor its in-house vertical market banking transaction application. In contrast, we found that AppManager is only somewhat configurable by customers, and PATROL is far behind eQ and AppManager in its ability to be tailored by customers.

In our tests, we found eQ Management Suite offered the best scalability as well as the best reliability of these three monitoring products. eQ's reasonable cost is icing on the cake.

Fixing the Problem

All three tools can send e-mail, emit an SNMP alert (trap) or page someone when a problem occurs. To fix the problem, each tool can also run corrective software or restart a failed service. A corrective action can consist of running a program, making network configuration changes, updating a database via certain SQL statements, copying a file or restarting an application or Windows NT/Windows 2000 service. All three tools integrate with network management systems such as OpenView, SPECTRUM, Tivoli and Unicenter TNG.

eQ's Solutions went beyond the problem fixing capabilities of AppManager and PATROL by also providing for dynamic modification of both Solution rules and variables. It exceeded our expectations by flexibly allowing us, with a few simple statements in eQ's powerful and sophisticated rule language, to enable, disable or reschedule other rules as well as modify rule variables and eQ internal settings.

eQ Management Suite, AppManager and PATROL all offer useful reports to help problem solvers and capacity planners stay ahead of the curve. Each can report on historical as well as current activity, both for problems detected and general levels of application and network activity.

Ease of Use

eQ has both a native Windows interface and a browser-based interface. The latter, called the Web Interface, is a well-designed and full-featured doorway into all eQ's functions. Through it, we were able to start and stop eQ Solutions, produce and examine reports (including graphs and charts depicting events and activities) and perform all the monitoring, administering, managing and reporting steps available through the native Windows interface.

PATROL augments its Enterprise Manager native user interface console with multiple Web interfaces, including one called Central Alerts - Web Edition (PCA) and another called Central Operator - Web Edition. PCA offers textual and graphic reports of PATROL activity as well as control over PATROL functions and behaviors. PATROL users view what BMC calls Business Views (displays of service processes) through the user interface. An administrator, if he or she wishes, can configure PCA to offer just a read-only view of PATROL activity for managers to see. Central Operator - Web Edition is the more detailed window into PATROL, showing specific information about the health of an application, server, operating system or computer. Its customizable views can include charts, graphs, gauges, text and ActiveX objects.

Another PATROL user interface module, Dashboard, is a Web-and-Java-based network performance management component that uses SNMP to discover and monitor network devices. Typically, administrators tell Dashboard to send e-mail reports in HTML format to particular recipients.

Like eQ Management Suite and PATROL, AppManager offers a native Windows and a Web-based user interface. AppManager's charting and reporting functions lack eQ's task-directed, context-sensitive guides, but are otherwise simple to use. To its credit, AppManager does have an advanced feature for troubleshooting performance and availability problems in which users can drag-and-drop what NetIQ calls Knowledge Scripts to obtain problem detail and graphical charts.

AppManager Diagnostic Console is a native Windows user interface through which users can remotely investigate and sometimes correct Windows server problems and errors. Getting information on Windows servers' current operational status and behavior, such as which files are presently being shared, is easy with Diagnostic Console.

All three products were easy to install and came with clear, comprehensive printed documentation.

Conclusion

Heroix's eQ Management Suite easily emerged victorious in our tests. It's a highly flexible, comprehensive application and network monitoring tool that excels at early problem detection, often fixes problems without human intervention, is robust, reliable and scalable, produces highly useful reports, is easy to use and is priced right. Solution Studio, which gives you the ability to flexibly tailor eQ Management Suite to your particular computing environment without having to become a programmer, is icing on the cake.

We recommend you give eQ a close look. Network administrators in your company will wonder how they ever got by without eQ, and we believe the tool will make their uptime and availability statistics shine.

Network Testing Labs' Testbed and Methodology
Our test environment for this review consisted of multiple Cisco-routed Fast Ethernet subnet domains and a T1 ISP connection. Our client and server platforms included Windows NT/98/2000, Unix (AIX 4.3), Red Hat Linux 6.2, Novell NetWare 5.1 and Macintosh System 8. Relational databases on the network were Oracle 8i, Sybase Adaptive Server 11.5 and Microsoft SQL Server 2000. Windows NT, Windows 2000 and NetWare shared files, while Internet Information Server (IIS), Netscape and Apache software served up Web pages. The network's protocols were TCP/IP, IPX, AppleTalk and SNA.

We ran the tools on three H-P/Compaq ML570 Proliant computers, each with four 900-Mhz Pentium processors, 2GB bytes of RAM and eight 9G bytes SCSI RAID drives. In each case, the operating system platform was Windows 2000 Advanced Server.

In our tests, we primarily looked for the ability to monitor the health and availability of our servers, operating systems, applications and network devices. The ability to resolve a problem automatically was a plus. We tested the sending of problem notifications by pager, e-mail and SNMP alerts (traps). We expected a product to produce reports that helped establish baselines, show current and historical server, application and operating system problems, identify trends and avoid future problems.

An Agilent Advisor protocol analyzer eavesdropped on the network traffic to reveal both overall utilization and the content detail of each tool's network traffic.

— Barry Nance
Network Testing Labs

About the Author
Barry Nance is a networking expert, magazine columnist, book author and application architect. He has 29 years experience with IT technologies, methodologies and products. Over the past dozen years, working on behalf of Network Testing Labs, he has evaluated thousands of hardware and software products for ComputerWorld, BYTE Magazine, Government Computer News, PC Magazine, Network Computing, Network World and many other publications. He's authored thousands of magazine articles and three popular books, Introduction to Networking (4th Edition), Network Programming in C and Client/Server LAN Programming.

He's also designed successful e-commerce Web-based applications, created database and network benchmark tools, written a variety of network diagnostic software utilities and developed a number of special-purpose networking protocols.

You can e-mail him at barryn@erols.com.

About Network Testing Labs
Network Testing Labs performs independent technology research and product evaluations. Its network laboratory connects myriads of types of computers and virtually every kind of network device in an ever-changing variety of ways. Its authors are networking experts who write clearly and plainly about complex technologies and products.

Network Testing Labs' experts have written hardware and software product reviews, state-of-the-art analyses, feature articles, in-depth technology workshops, cover stories, buyer's guides and in-depth technology outlooks. Our experts have spoken on a number of topics at PC Expo and other venues. In addition, they've created industry standard network benchmark software, database benchmark software and network diagnostic utilities.