Electronic batch records: An MES pilot implementation case study

Headnote

This article describes the implementation of a pilotscale manufacturing execution system (MES) within a commercial biotechnology company. The authors focus on-the reasons for the pilot-scale implementation, the importance of the recent electronic signature ruling, the rationale behind the final vendor and MIES selection, and the system's architecture and interfaces. An in-depth review of the selected MIES functionality and an update on the project are also provided. The factory acceptance testing, site acceptance testing, and validation aspects of the system are intentionally not covered because of the defined scope of the article.

This article describes the implementation of a pilotscale manufacturing execution system (MES) within a biotechnology company. The company (IDEC Pharmaceuticals Corp., San Diego, CA) develops targeted immunotherapies for cancer and autoimmune diseases. Many of the company's products are immunologically active antibodies that are designed to target immune system cells called lymphocytes and harness the body's own mechanisms for fighting disease. The company also is developing a small-molecule anticancer agent that acts through a different mechanism than most anticancer drugs and that has shown broad-spectrum activity against a variety of tumor types.

THE PROJECT

IDEC currently has completed conceptual and preliminary engineering design exercises for a commercial large-scale manufacturing facility. The facility's design is based on longterm corporate strategies and on material requirement estimates for products currently in the development pipeline. The primary automation objective for the new facility is the control of the production, materials, utilities, and building environments through an integrated plantwide control system. The control system will provide overall process-plant and utilitysystem monitoring, batch operation control, supervisory control, and data acquisition. The control system also will be fully integrated with an MES and other high-level business systems such as a laboratory information management system (LIMS), an enterprise resource planning (ERP) system, and an electronic document management system (EDMS).

The design, construction, start-up, and validation of such a facility are not trivial tasks. The facility's operation is highly dependent on its electronic systems. IDEC considered as invaluable any information that could be acquired regarding integration and operation of the various electronic systems within such a facility before or in conjunction with the detailed design exercise. An opportunity to initiate a pilot MES implementation project presented itself within IDEC's current manufacturing operations, specifically within the clinical manufacturing area (CMA) and materials control. The intent was to split the project into phases and, during these phases, to gain valuable implementation, start-up, and operational information for an electronic system of this nature.

The first phase aimed to provide a client-server-based electronic batch record (EBR) system that used as much as possible the infrastructure currently in place at IDEC and to add new components to this infrastructure as necessary. The deliverables for this phase focused on providing and using the material tracking, work in progress (WIP) and availability reporting, material reconciliation, process data management, lot genealogy, EBR reporting, and bar code capabilities of a standard MES. It was important to deliver a functional interface between the MES and the existing ERP system (Ross, Atlanta, GA) installed at IDEC because the ERP system contained master material inventory and initial recipe and formula definitions.

The direction for the second phase and subsequent work consisted of taking advantage of other functions and interfaces within a standard MES. However, a number of these activities required capital investment from IDEC. The list of activities for this phase included directly tying in process data to the EBR system through supervisory control and data acquisition package interfacing, using the computer as a second verifier for weighing operations by interfacing to weigh scales, and interfacing to existing maintenance management systems for retrieval of equipment calibration records.

IMPORTANCE OF THE ELECTRONIC RECORDS RULING

In 1991, members of the pharmaceutical industry met with FDA to determine how they could accommodate paperless record systems under current good manufacturing practice (CGMP) regulations. The authors do not intend to discuss in detail the history behind the final ruling, but wish to note that effective 20 August 1997 the final rule provided the criteria under which FDA considers electronic records to be equivalent to paper records and electronic signatures to be equivalent to handwritten signatures.

An electronic signature as defined by 21 CFR 11 is "... a computer data compilation of any symbol or series of symbols executed, adopted, or authorized by an individual to be the legally binding equivalent of the individual's handwritten signature" (1,2).

After 6 years of discussions, task force meetings, and reports, this ruling finally allowed FDA-regulated industries to take full advantage of and to use electronic record keeping to its full potential. The regulation gives manufacturers a more effective and efficient means for continuous improvement of their manufacturing operations than was possible with paper-based systems. This in turn lowers the cost of goods, allows products to be introduced (lot release) to market in a shorter time, improves record keeping, and increases the quality and comprehensiveness of security, data integrity, and audit trails. In 1994, MESA International (Pittsburgh, PA) conducted an academic study of the substantial benefits of an MES with respect to the manufacturing environment (3). MESA International cited examples such as reduced manufacturing cycle, data entry, and lead times; reduced WIP and paperwork between shifts; improvement in product quality; and the elimination of lost paperwork. The FDA electronic signature ruling was the last stumbling block to making these benefits available to FDA-regulated industries. How important are these benefits and what dollar value can be can be assigned to them for a competitive drug manufacturing operation in the late 20th century and into the 21st century?

VENDOR AND MES SELECTION

In late 1995, IDEC organized a small group to search for possible suppliers and systems for its MES solution. The group comprised representatives from manufacturing (materials control and operations), management information systems (MIS), and an executive-level member. The company also hired an experienced consultant to help the group formulate, clarify, and document views and develop a requirements document.

During the next 2 years, the group identified four vendors with MES solutions that they considered worth further investigation. On-site demonstrations were performed, and it became clear from IDEC's perspective that three of the four vendors had at least one of two major faults:

* The MES technology was incompatible with IDEC's current and future technology direction and standards.

* Implementation of the MES required IDEC to reengineer a number of current business practices, procedures, or, worse, manufacturing processes. It appeared that the system had been engineered from a top-down approach (i.e., from a finance and business decision level rather than a manufacturing process level).

The final vendor was contacted for a formal proposal and site visit. The vendor was already a major player in the areas of power, process control, instrumentation, and MES for the pharmaceutical and biotechnology industries. Its MES solution had been designed, implemented, and supported by engineers with many years of experience in the process control field (programmable logic controller and distributed control systems [DCS]). Experts in this field usually are given a defined (regulated) manufacturing process and are asked to mold a system solution around this process (i.e., although the way the manufacturing operations are documented and performed would change, the fundamental intent and execution of the operations would remain the same).

An operating MES site visit was performed with an expanded IDEC team to involve other disciplines that would have a significant impact on the project. This enabled the smaller IDEC core team and CMA operations and validation personnel to see the system in an operational manufacturing environment. The site visited had not only a functional MES, but also integrated and functional interfaces between the MES and an ERP system, EDMS, CDmastering and archival system, and DCS (see Figure 1).

The systems enclosed in clear boxes in Figure I were purchased, off-the-shelf applications; those shown in shaded boxes were developed by the MES vendor. The MES vendor provided the developed applications as well as the software necessary to integrate the various systems.

It was clear that the MES vendor, ABB Industrial Systems (West Henrietta, NY), had captured the potential of automation and advanced computer technologies in its development of an EBR system as demonstrated at the external site and the IDEC demonstrations, and by its understanding of IDEC's required solution that was documented in its complete MES proposal. After the expanded IDEC team returned from the site visit, a relatively short time passed during which details of the proposal were clarified with ABB, a confidentiality agreement was put into place, an audit date of the ABB Houston office was agreed upon, project teams were finalized, and a kick-off date was selected. A contract engaging ABB to supply an MES to IDEC for the CMA and materials control operations was formally signed in October 1997.

MES ARCHITECTURE AND INTERFACE DEFINITION

An important goal of the first phase was to deliver a functional interface between the MES and the existing ERP system installed at IDEC. This required an MES solution with three functional areas. The first functional area provides the interface between the ERP system and the MES solution. This area allows the transfer of read-only ERP information to the configurable MES solution. An IDEC consultant developed ERP views for the transfer of the ERP system data. A functional specification was developed to include the ERP database table-mapping structure with the defined relationships of the ERP database tables for warehousereceiving functions (quarantine and release), weigh and dispense (W&D), the label mechanism, and the job or batch release and complete mechanism. A schematic of the interface solution is shown in Figure 2.

The second functional area is the configurable MES solution, which will be discussed later. The last functional area is the interface between the MES and the ERP system, which provides inventory transaction updates to the ERP system. The MES does not write the inventory transactions to the ERP database tables directly; instead, it writes to intermediate ERP database tables. The data are then transferred to the main ERP database tables after appropriate data integrity checking on the ERP system. The inventory transactions are the actual raw materials consumed and the actual products produced.

The implementation of this interface structure created a single and unique master data source for master product, master lot, batch, vendor, and recipe and formula definitions derived from the master ERP database tables and transferred into the MES. This interface concept requires maintenance of the master data sources on the ERP system and enables IDEC to eliminate customization of the base MES functionality.

A second functional specification document was developed to define the hardware and system software for the MES. The system architecture drawing is shown in Figure 3. Descriptions of the components of the system architecture follow.

ERP system, The ERP system controls and tracks inventory, recipes, item lists, item units, receipts, and batch-scheduling information. This information is read and used by the configuration client, receiving client, W&D client, and the batch activity manager (BAM) server. Actual consumption is made available from the BAM server.

Quarantine client. A quarantine client in the warehouse is used to release materials from quarantine and to print a released label on a networked bar code label printer.

Receiving client. A receiving client in the warehouse is used to receive materials into quarantine and to print quarantine labels on a networked bar code label printer.

Configuration client. Each batch production record uses a preconfigured W&D procedure and a preconfigured manufacturing procedure. These procedures are defined on the configuration client and stored on the BAM server, Various reports of inprocess and completed manufacturing runs are requested from this workstation.

W&D client. A W&D client in the preweigh area is used to record the requisitioning of materials against a scheduled procedure's preconfigured bill of materials or against a department or equipment number. After a weighing or dispensing activity for an item is completed, a label is printed on a networked bar code printer.

Cell culture and purification operator stations. Operator stations are used to execute the preconfigured manufacturing procedure when an operation is scheduled for execution. The operator stations are hand-held, portable terminals with bar code scanners; communication is accomplished via a radio link. This allows operators to move about the plant and record information directly into the EBR system during the manufacturing process. Labels generated by the manufacturing procedures are printed on the networked bar code printers in the manufacturing areas. In effect, the paper batch record has been replaced with electronic data entry screens and radio frequency (RF) portable bar code scanners. .

BAM server. The BAM server stores the database and applications associated with W&D activities, electronic batch recording, and reporting.

Radio controller. The radio controller links the hand-held operator stations in cell culture and purification to the network for communication with the BAM server.

Bar code printer. Five networked bar code printers are included in the current configuration for label printing. Figure 4 is an example of a typical label format.

Other PCs. The BAM server uses an Oracle 7 database. Individuals on the network with password access and installed ODBC drivers can access and use production information in various PC applications such as Access and Excel.

Existing network printers. All reports are printed to the default Windows printer configured on the client from which the report is being printed.

Existing network. The current IDEC network uses the TCP/IP protocol for communication between nodes. The network is 100 MB Ethernet, and network connections are 10 BaseT.

Existing network file server. The MES applications run by the configuration client, W&D client, quarantine client, and receiving client are stored on this server.

The MES application consists of two modules: a W&D module and an EBR and BAM module. Each module has a separate configuration and execution task.

W&D MODULE

All IDEC manufacturing processes require that all raw materials needed in commercial or clinical production batches be weighed and dispensed before production begins. The W&D operation is driven by the materials section of the batch record, which in essence is a pick list for all raw materials required by the batch. The W&D screens within the MES have been developed to integrate bar code readers, weigh scales, and ERP system information as available or as integration permits.

The W&D operation allows operators to configure a reusable procedure that guides the operator through the sequential W&D steps of materials for a batch (see Figure 5). Configuration of the module is performed using a graphical user interface (GUI); in IDEC's MES implementation, this is a PC client. The W&D application downloads the formula, bill of materials, and batch identifier information from the ERP system. The operator executes a W&D procedure while interfacing with a local PC. The PC is equipped with a hand-held bar code reader for input of bar coded information such as operator, equipment, and material identification. Normal data entry, as required, is performed via the PC screen and is prompted by displayed procedural instructions. Materials allocated to a specific batch are identified with a label that typically includes both human-readable text and bar code data.

The execution phase is the process of weighing, measuring, and dispensing materials for a specific batch. The operator initiates the W&D procedure by selecting from a table of scheduled batches for an area that requires a W&D activity. Starting a procedure invokes the material screen, which displays all the configured items for the batch and their status. In simplistic terms, as part of the W&D procedure, the system instructs the operator to tare the scale, weigh the item in the package sizes configured, and generate a label for each package. The system tallies and validates the material quantities and the material identifications with the master files required for the batch.

Descriptions of the key features of the W&D module follow.

* As the W&D procedure downloads the formula from the ERP system, the bill of materials and batch identifier information is maintained only in one system and is shared. This reduces data input errors and shortens data entry and data maintenance time.

* The formula downloaded from the ERP system automatically provides a pick list in the W&D module for the batch.

* The W&D module in effect contains on-line procedures for the W&D operations. Appropriate personnel validate and approve these procedures before releasing them for manufacturing operation use.

* Bar code labels are printed automatically after each weighing operation is completed.

* In-process material use can be tracked and reconciled to individual lots using the bar coded in-process labels.

* The W&D application monitors and tallies quantities weighed and dispensed to a batch. This check ensures that the weight for each material is within the established specification and tolerance levels. The MES will warn an operator if the quantity's weight exceeds the specification plus tolerance limit. If this occurs, a deviation report must be completed to finish the operation if an out-of-tolerance amount is dispensed. The status of the ingredient remains as "in process" if the specified quantity is not met. In addition to incorporating weight tolerance and, if required, identity of the specific scale, the W&D module has the capability of handling items that require counting or measuring without scale interaction (see Figure 6).

* The system tracks and monitors partial issues and mixed lots of materials in one or more containers. An operator is required to indicate which containers include material from two or more raw material lots.

* Invalid material or expired material cannot be processed within a W&D operation. The operator is prohibited from proceeding after the bar code is scanned and the material is identified as invalid or expired.

* Material requisition capability is supplied, which permits the dispensing of materials directly to equipment and departments.

* System security requires that operators scan their identification badge or type in their identification and supply a password to authorize and/or confirm operations to the W&D module. All required confirmations and authorizations are captured in the EBR.

* With the integration of weigh scales and the computer acting as the second verifier, the immediate benefits are the elimination of transposition errors and a significant reduction of operator data entries.

* The W&D module meets all CGMP requirements for its intended operation including 21 CFR 211.69 (automatic, mechanical, and electronic equipment), 211. 101 (charge-in of components), and 211.188 (batch production and control records).

EBR AND BAM MODULE

This module allows an operator to configure a reusable procedure that guides the operator through the sequential steps in a manufacturing procedure or batch record. Configuration of the module is performed using a GUI; in IDEC's MES implementation, this is a PC client. The procedural results, the EBR, are recorded into a relational Oracle 7 database, which can be configured to allow access by other systems. If implemented and integrated with the MES, the EBR can be written and stored into an EDMS and, from this system, can be written to a CD-ROM for permanent archival or secure, tamper-proof transmittal (see Figure 1).

The BAM concept involves the generation of specific transactions for the collection and display of process and event data. During the configuration phase, the transactions are configured with the manufacturing product and equipment information and are assembled into an executable sequence. Each transaction uses forms for configuration and subsequent data entry. Transactions are analogous to the steps in a procedure or the pages of a paper document. The transactions are defined within the BAM and include steps such as verification of equipment status, scheduled sampling, manual operation, routine data entry, item addition, item move, and recording of event start (see Figure 7). Currently there are 32 distinct transactions that allow the modeling of a paper batch record in terms of manual and automated activities. The academic basis behind these transactions is the Purdue Enterprise Reference Architecture (PERA) authored by Dr. Theodore J. Williams of Purdue University (4).

PERA supports and guides the development of the master plan for an enterprise business entity. The methodology covers the complete project of introduction, implementation, and operation of an enterprise business entity, which may be either part of a larger entity or the complete enterprise itself. The overall applicability of PERA depends to a great extent on the development of a set of generic tasks, functions, and macrofunctions to describe an enterprise integration system or indeed any enterprise.

This system design, through the use of PERA, ensures a precise fit to each manufacturing batch process and provides consistency of operational execution for each completed procedure. If necessary, transactions can be configured to require two electronic signatures at the completion of a step.

The module also allows for backward and forward linking of transactions to set up dependencies between transactions previously executed and those scheduled to be executed in future batch records. Instructions, in effect electronic standard operating procedures, can be configured with each transaction. The MES solution contains utilities that permit the modification of existing electronic procedures and the creation of new procedures with minimal effort.

Execution of the electronic batch procedure is accomplished by the use of a character-based operator interface. This interface may be run from any terminal emulation device including a PC or portable bar code scanning device with RF capability. The system guides the operator through the various sequential steps involved in the manufacturing process.

The operator initiates procedure execution by selecting a batch run number - defined and supplied by the ERP system - that is associated with the manufacturing area to which the operator is assigned. The operator receives, in sequential order, electronic batch instructions that mimic the instructions of the original paper batch record (see Figure 8). The system validates operator actions and data entry, and any transaction data outside the range of the configured instruction limits require that the operator enter an exception and deviation comment before proceeding (see Figure 9). Each transaction allows the operator to enter a comment if desired.

Lot and batch relationships are maintained with respect to material consumption and batch production at specific manufacturing steps for each batch. The final EBR lists all personnel that worked on the batch, equipment used, material consumed by lot number and quantity, material summary and detail, each transaction or step instruction performed in sequential order, process data, and an exception report.

A number of the W&D module's key features also apply to the EBR and BAM module and will not be repeated, particularly those pertaining to system security and CGMP compliance. Other key features of the EBR and BAM module follow.

* Configurable transactions allow enough flexibility of procedure creation to enable a precise fit to the manufacturing process.

* The use of standard Oracle forms, pick lists, and a fill-in-theblank configuration methodology eliminates any requirement for custom programming.

* The configuration phase enables the EBR to be constructed in sequential steps that are analogous to the steps of the paper batch record with which operating personnel are very familiar.

* Existing procedures can be modified and new batch procedures can be created with minimal effort.

* A higher level of operational consistency and quality from batch to batch is achieved because the EBR requires that the operator follow the sequential steps in the batch record. This also facilitates streamlining shift changeover activities.

* Data collection can be validated at the time of collection. The combination of data validation techniques with bar code reader technology significantly reduces the number of data entry errors. Any data outside predetermined limits requires that a deviation and exception be written at the time of occurrence.

* The time for batch record approval and lot release is reduced. If correctly configured, a batch or lot cannot be completed unless all of the pertinent information is captured, including process data, electronic signatures, confirmation electronic signatures, deviation reports, and operator comments. Theoretically, if a batch is completed with all data captured correctly and no deviations reported, then the batch or lot can be released from manufacturing electronically and instantaneously without the need for human review.

MES UTILITIES

Various utilities exist as part of the base MES functionality to enhance the system's performance and allow system configuration, administration, and implementation in the shortest time possible. These utilities include an archive function, audit trail capabilities, copy procedure tools for both W&D and BAM modules, and user administration capabilities.

The archive application allows a user to archive batch run information to a text file, to retrieve batch run information from a previously archived text file, and to delete a completed batch run file.

An audit trail is maintained for each database table in the BAM and EBR module and the W&D module. Each time a row in a database table is deleted or updated, a database trigger automatically records the changes in the audit trail database table.

The "copy BAW and "copy W&D" applications allow a user to copy all information about a specific BAM or W&D configuration from a configured formula to a new formula defined in the ERP system.

A user administration application allows the system manager to easily maintain user information, passwords, and roles. From this application, the system manager can create new users, change passwords for users, and grant roles to users.

PROJECT UPDATE

At the time this article was written, the first phase of the engineering and implementation of the MES pilot had been completed. The system as shown in Figure 3 had been installed at IDEC, and a formal site acceptance test (SAT) had been completed. The MES/ERP interface had been implemented as shown in Figure 2, and a formal SAT that demonstrated the functional integrity of the design had been completed for the interface.

IDEC personnel from materials control and CMA are currently breaking down the various batch records into the required sequential steps before entering them into the system. A critical mass of data entry into the system is required from the configuration exercise before the MES can run in parallel with existing paper operations. The MES will run in parallel with existing paper documents until the validation effort is completed and IDEC personnel have had enough operational experience with the new system to make an error-free transition to the MES as the primary system. Development of validation protocols for the MES is in progress; validation protocol execution will occur immediately after completion of an internal business network characterization project that is currently underway.

The second phase of the pilot has begun. This phase will concentrate on adding further functional enhancements to both the base MES system and the ERP interface. Expansion of the current pilot to the commercial manufacturing facility as a secondary system is under consideration.

ACKNOWLEDGMENT

The authors would like to thank the following individuals and groups for the efforts they made to ensure the successful completion of the first phase of the project: Chris Burman (executive sponsor), Basim Hilal (CMA), Jesse Ivers (MIS), Kevin Kelley (MIS), Saya Mehta (materials control), Laurie Sangalang (materials control), all with IDEC; David Wilkinson of TTGI (consultant for Ross); and all ABB Industrial Systems MES groups based in Houston, Texas, and Rochester, New York.

References

REFERENCES

References

1. Code of Federal Regulations, Title 21, Food and Drugs (General Services Administration, Washington, DC, I April 1973), Parts 11, 210, and 211.

References

2. C. Deitz, J. Nelson, and J. Salazar, "New FDA Regulations Using Computer Technology," Pharm. Eng. (Jan.-Feb. 1998).

3. MESA International, "White Paper Number 1. The Benefits of MES: A Report from The Field," 1994.

4. T.J. Williams, "The Purdue Enterprise Reference Architecture," Computers in Industry 24, (1994).

ADDITIONAL READING

References

* S. Zafar Kamal, "Integrated Enterprise Proves Key to Flexible Manufacturing," Intech (July 1998).

* R. Martin, "The Dis-Integrated World of Quality Management," AMR Report on Manufacturing (May 1998).rl