Friday, October 2, 2009

Job Scheduling Maze in Distributed IT Landscapes

We certainly learn new things every day, and sometimes out of pure serendipity. Namely, when I was recently asked by one of my industry contacts (working for a PR agency) whether I would like to have a briefing with and about his client, whose name included the word “batch” as a part, I agreed, thinking it was a process enterprise resource planning (ERP) vendor or maybe a manufacturing execution system (MES) vendor.

To my chagrin, as soon as the Web conference meeting and demo started, I realized that I was in the quite unfamiliar territory of enterprise job scheduling and workload automation, especially when it comes to highly diverse and distributed information technology (IT) environments. Many of these jobs are still run in a batch mode without human interaction and intervention (at least preferably).

Some batch job examples would be: database/data warehouse updates, payroll runs, file copying and archiving, systems rebooting, disks de-fragmenting, reports printing, processing insurance claims, billing statements, and/or enrollments, file transfer protocols (FTP), and so on. Thus came the ActiveBatch product name, I guess.

For someone who is not that technically proficient, the knee-jerk reaction was to say: “Sorry, this was a misunderstanding”, but I was drawn in by a good discussion and the product’s apparent usefulness. In fact, the briefing made me think about how often we take things for granted, and how unaware we are of the legwork that IT staff, especially within large corporations, is conducting behind the scenes. We are usually aware of the IT folks only when our PC is not working or the email server is down, and when we “need help now!”

The conversation made me realize that this vendor truly understands mission-critical business that requires high-performance systems. ActiveBatch originators know what it means to meet both deadlines and service level response times. Conversely, while the commonly used enterprise applications, databases, platforms (e.g., Microsoft Windows with Task Scheduler or UNIX with Cron) contain limited scheduling functions, these address only basic requirements within the confines of the individual system.

In fact, from my erstwhile experiences as a Baan (now Infor ERP LN) and SAP R/3 functional consultant, I vaguely recall some task scheduling functions (e.g., setting overnight or weekend material requirements planning [MRP] runs, or performing a trial balance and general ledger [G/L] updates at a certain time). The technical consultants would set up these batch jobs within the Baan Tools or SAP Basis administrative capabilities.

Overcoming Individual Systems’ “Autism”

However, the problem is that as the number of systems, applications, databases, and whatnot platforms increases, the IT business community requires automation among and across these various systems in an end-to-end manner to provide a single point of workload automation. In addition to the ability to integrate all the pieces of a heterogeneous environment, such a system should be able to schedule jobs based on events and associated built-in business logic, as opposed to merely on a set date/time, and in a single-path manner within a silo (as is the case with inwardly-oriented “autistic” enterprise systems).

To that end, ActiveBatch provides a central point of scheduling that allows each of these disparate systems to be automated and integrated into coherent workflows. The essence of ActiveBatch, Intelligent Automation, is to provide the level of integration for applications, platforms, databases, and specific functions without the need for costly and tedious code scripting (and reliance on programmers).

What prospective customers look to ActiveBatch to accomplish is the following:

* Improve IT service levels;
* Integrate workflows and business processes between diverse applications and platforms;
* Reduce the number of manual errors (which otherwise come with the territory of relying on humans);
* Implement a centralized view of jobs that span across the vast platforms’ landscape; and
* Eliminate an “artificial” wait or idle time that has to be built into existing workflows (to accommodate all imperfections).

In a nutshell, this is about ultimately reducing the cost of IT operations. Some customers cite the fact that often in the past the original enterprise job scheduler (typically made in-house in a heavily customized manner) would allow some jobs to fail without notifying anyone, and the company wouldn’t know anything about it until someone belatedly complained about not getting the output of the job (e.g., a report). Tracing and fixing these instances would often take more time and resources than the job scheduler was supposed to save in the first place.

ActiveBatch and Advanced Systems Concepts Inc. (ASCI)

But let me backtrack and talk about the ActiveBatch’s genesis. The company that owns ActiveBatch is privately held Advanced Systems Concepts, Inc. (ASCI), with headquarters in Morristown, New Jersey (NJ), United States (US). It was founded in 1981 as a system software engineering and consulting company focused on the development of products for former Digital Equipment Corporation’s (DEC) OpenVMS operating system (OS) product (meanwhile of course, DEC was acquired by COMPAQ, and now both are part of Hewlett-Packard [HP]).

ASCI’s first product, INTACT, was a transaction processing system for OpenVMS to allow customers to use OpenVMS systems in commercial applications. INTACT was licensed to major financial organizations around the world. In late 1986/early 1987, DEC exclusively licensed INTACT from ASCI, and renamed it DECIntact, as a solution incorporated as part of its erstwhile Enterprise Application Strategy. DEC’s decision at the time was based on the need to compete with IBM and its transaction processing system, Complex Instruction Computer Set (CICS), with a counterpart competitive solution. INTACT and DECIntact are both still in use in many organizations around the world.

ASCI has also developed other layered product solutions for the OpenVMS market including:

* SHADOW – the first shadowing/data replication system for OpenVMS;
* WATCH – for help desk and other similar functions that allow one user to watch other terminal sessions;
* Performance Simulation System (PSS) – the automated regression and application testing system for the OpenVMS applications; and
* VIRTUOSO – a virtual container technology that enables the development of virtual disks to be used as random access memory (RAM) and cached disks for performance improvements, encrypted disks for security, and more.

In 1991, ASCI enhanced SHADOW with a new family of products including FileSHADOW and RemoteSHADOW for OpenVMS. RemoteSHADOW for OpenVMS has since been installed in over 1,000 customer environments to reduce the time of data recovery in the event of a system or site loss. For instance, in 2001, during the dreadful 9/11 attacks, RemoteSHADOW for OpenVMS was used by many financial organizations, including Dresdner Kleinwort Wasserstein (DSW) Bank, to recover their data and get their business functioning again within hours at an alternate site.

In 1996, ASCI broadened its OS focus from exclusively OpenVMS to include UNIX and Microsoft Windows. One of the UNIX-based products that is still in use is DeviceShare. For its part, XLNT (Extended Language for NT), a command and scripting language for Windows (not only Windows NT, as the name would imply), was introduced to offer system administrators a scripting alternative to managing systems and developing workflow scripts. Over 100,000 XLNT licenses are currently in use around the world.

Enter ActiveBatch

In 1998, XLNT users needed a batch system to automate and run their XLNT scripts on a schedule. ASCI thus introduced Batch Queue Management System (BQMS) to assist XLNT users to automate their scripts on and across Windows servers. In 2000, BQMS was renamed and re-introduced as ActiveBatch V3, a heterogeneous job scheduling solution for Windows, UNIX , Linux, and OpenVMS systems.

ASCI proudly claims to be self-funded, and that its development, quality assurance (QA), and support teams are its own (and not outsourced or off-shored). The company has licensed ActiveBatch to over 1,400 customers in 34 countries around the world.

With its Intelligent Automation capabilities and performance, and having been tested across 2,000 servers, performing over 1,3 million jobs per day, ActiveBatch is fast becoming the Workload Automation and Enterprise Job Scheduling solution of choice. Additionally, ASCI has licensed nearly 4,000 clients in 34 countries around the globe for the full range of its products. Its clients include many of the Fortune 1000 companies with a mix of medium to large enterprises.

ASCI competes with several powerful and renowned application providers in the Workload Automation and Job Scheduling market including Computer Associates (CA) Unicenter AutoSys, BMC CONTROL-M, and IBM Tivoli. Many of these vendors’ products were originally developed for the mainframe, not necessarily for today’s heterogeneous, horizontal server environments. As a result, they have customarily been adapted (retrofitted) to today’s distributed server environments.

More modern job scheduling vendors’ products like Tidal Software, UC4, Redwood Software (especially in SAP environments), Quartz (open source), and of course ActiveBatch, understand the distributed server environments, and have targeted their solutions to address this requirement. Part 2 of this blog series will analyze the ActiveBatch architecture and evolution in terms of functional and technical capabilities.

No comments:

Post a Comment