Columbia Accident Investigation Board Public Hearing Monday, April 7, 2003 1:00 p.m. Hilton Hotel 3000 NASA Road 1 Houston, Texas Board Members Present: Admiral Hal Gehman Major General Ken Hess Brigadier General Duane Deal Dr. Sally Ride Dr. John Logsdon Mr. G. Scott Hubbard Mr. Steven Wallace Witnesses Testifying: Col. James Halsell, Jr. Mr. Robert Castle, Jr. Mr. J. Scott Sparks Mr. Lee Foster ADM. GEHMAN: Good afternoon, ladies and gentlemen. This public hearing of the Columbia Accident Investigation Board is in session. We're privileged to have with us today two experts to help us see our way through some of the issues that we have to deal with, and we're going to deal with the treatment of anomalies and waivers and certifications and all that sort of stuff today. We have a panel of two -- I don't know if you'd call them experts or not; we'll see at the end of the day whether they're experts or not -- but to help guide us through the first part of this process. The first is Colonel James Halsell, who is an astronaut and has a couple of duties, one of which is, I presume, to command a mission here in the future, I trust; and Robert Castle, who is from the Mission Operations Directorate. Gentlemen, before we begin, let me ask you to first to affirm that the information you provide the board today will be accurate and complete, to the best of your current knowledge and belief. THE WITNESSES: I do affirm. JAMES HALSELL and ROBERT CASTLE, JR. testified as follows: ADM. GEHMAN: Would either one of you start and introduce yourselves and tell us a little bit about your background but also tell us what your duties are today. COL. HALSELL: Okay. I'll start first, sir. It's my privilege to be here to have the opportunity to work toward what certainly anybody at NASA considers to be one of the most important things we'll ever do in our career -- that is, to find out what happened, to fix it, and get back to flying safely. I have a background in the Air Force. I'm an active duty Colonel in the Air Force. My background in aviation was fighter aviation, followed by test aviation, and then an assignment to NASA for the last 13 years, since 1990 as an astronaut. I had the privilege of flying five missions; and at the conclusion of my fifth mission, I was asked to take on, as a career-broadening experience, a management job down at the Kennedy Space Center as a launch integration manager, working directly for the program manager, Mr. Ron Dittemore. I did that from the summer of 2000 until January of this year, when I was relieved of that job in order to take my next assignment, which was to command STS 120, which will be a mission to the International Space Station, taking up Node 2, one of the hardware components that will complete the American initial phase of the construction of the station. If you'd like, at this point in time I can talk to you -- ADM. GEHMAN: Before we do, let me ask. Do you also have a role in the return-to-flight process? COL. HALSELL: Yes, sir. I received word just two weeks ago that I would be requested to head up a return-to-flight planning team. We would be doing a staff planning function, reporting directly to the deputy associate administrator for station and shuttle. That's Retired General Michael Kostelnik. Our job is to be his interface to the shuttle program and, in fact, throughout the NASA system working this issue, to come forward with recommendations and options in response to the Columbia Accident Investigation Board's findings and recommendations. So the way it should work is that once your investigation board wraps up with a report, and hopefully even in the interim before that final phase, we'll have the opportunity to map out a response to your investigation board's findings and recommendations. I'm sure that we'll come down to a set of options that we'll offer up to our leadership and our management and they will make some of the tough choices that have to be made with regard to what has to be done to fly safely again, what needs to be done in the long term to make the system even safer. ADM. GEHMAN: Let's let Mr. Castle introduce himself, and then you can start. MR. CASTLE: Okay. I'm very honored to be here and take part in this, in the return-to-flight effort for the Columbia. A little bit about myself. I'm a full-time career civil servant. I've been working for NASA for 25 years now. I started working one of the mission control sections as a communications officer, did that for about ten years, and then was a mid-level manager for about a year and then was selected for the flight director office in 1988. So I've spent right at 15 years as a NASA flight director, running missions in Mission Control. I have recently left that job to become the Missions Operations Directorate chief engineer and currently working on things like orbital space plane and some upgrades in the control center as well as contributing work on the International Space Station. I should also say I was a shuttle flight director for virtually all of that time. The last two years or so, I've switched over and become mainly a flight director on the International Space Station. That started around the middle of the year 2000 was when I did that much more than I did shuttle flights. So that's my current role to date. ADM. GEHMAN: Thank you very much. Colonel Halsell, if you have a statement or perhaps a presentation, we're ready to listen. COL. HALSELL: Yes, sir. I did come prepared with a presentation package. Certainly I would expect -- and feel free, as I'm sure you will, to ask me questions as we go along in this somewhat lengthy package. It's my understanding that I've been asked here today to give you any information that I might provide with the preflight process. In the shuttle program we call it the Flight Preparation Process, FPP for short. So if I use that acronym, that will be what I'm talking about. And that is the all-encompassing phrase, if you will, for everything that we do to get ready to go fly safely, including a subpart of that is the Certification of Flight Readiness and all the reviews and boards that we go through for that. Before I launch off into the details, it might be helpful if we just review the basics. The basics are basically this. The way the shuttle program is set up -- and I believe correctly and appropriately so -- is we have a set of requirements. It is huge, long list of requirements. It's broken down by the projects and the elements and all the contributing manufacturers, but the space shuttle program is responsible to be the keeper of the list of requirements. It tells us how we're going to build a component, how we're going to use it. It tells us how we train the crews. It tells us how we prepare the vehicles. Everything we do answers back to a requirement; and before we go launch a shuttle mission, it's absolutely required that we know we have lived up to and, in a closed-loop accounting fashion, answered each and every one of those requirements successfully. In a perfect world, you would have your requirements on one hand and before we go to launch, you'd have absolute and utter proof that you met each and every one of your requirements. We do live in that perfect world except there is such a thing as a waiver, in the sense that oftentimes if you can't meet the intent, indeed, the scripture of a requirement, then you have to come forward to the program, and specifically the program manager, and make the case for what you are offering instead is sufficient to allow a complete productive and safe mission. If you can pass that test, then with the waiver we are allowed to go ahead and fly. So it's requirements, closed-loop accounting system, and to the degree to which they don't match up perfectly, we enter into the waiver process. That's the 37,000-foot view of what we do, and almost everything that we talk about from this point on could be tied back to that very simple basic process. I know that after Challenger, it was recognized that these processes were not as disciplined and as rigorous as they should be; and I believe what I hope to tell you today and what I hope comes out of my presentation is that following the Challenger disaster, we went back and did rigorously enforce that discipline. In the degree to which we fell short in the Columbia accident, that's why we're here today and that's what we want to find out. I think it might be helpful just to lay out a couple of other basic thoughts. The shuttle was designed with the philosophy that you should not have a system in which you suffer a failure and you lose your vehicle or your crew. It needs to be fail-safe. Furthermore there was a high operational desire to be fail-operational -- that is, suffer a failure and still complete the mission. The basic requirements are that the vehicle and all of its subsystems will be fail-safe. From the very beginning, there were three of the systems which it was acknowledged we could not achieve that desired goal. The thermal protection system was one. It was recognized as being a Criticality 1 -- that is, if it doesn't work, you're going to lose the vehicle and/or the crew and we don't have a backup system to it. Pressure vessels, whether it's the pressure vessel in which the crew resides or the pressure vessels which holds our fuels and our oxidizers and our cryogens, was another. And finally the primary structure of the vehicle. The vehicle was not built with the intent that you could lose anyone -- you could always guarantee that you could lose one primary load-bearing piece of the structure and still maintain your safety margins. So those are the three areas where the design of the vehicle, it was acknowledged, would not live up to the basic requirement of being fail-safe. On the other hand, in the area of avionics, they designed it with a higher than fail-safe, that is, a fail-operational requirement. In our avionics area, it was designed to be able to suffer any one failure and continue to nominal end of mission. Those are my opening thoughts and maybe background that might help you as we delve down into the flight preparation process in detail. So with that, if I can press on to the next slide, please. This is a flow chart that shows you the program level reviews. Each of these represents a review, a large meeting of all the relevant NASA and contractor personnel; and it's also just a program level. Below each of these program level reviews is a vast array of project level reviews, but let me just briefly go through this and it will give you the outline of what we do and how we do it. Starting in the upper left-hand corner, the Flight Definition Requirements Document. That is the bible that a flight, a mission, in the preparation of a vehicle for that mission, where it all gets laid out. Normally this is presented to the Program Requirements Change Board, which is the program manager's venue for considering these top-level issues, about 16 months prior to flight. You can go from the front of the vehicle to the tail of the vehicle and talk about the level of detail, but basically that first block should be preceded by two or three years of preceding blocks where our customer and flight integration office receives inquiries from our potential customers to understand what payloads they want to fly, what mission requirements they are considering, and that's mapping those against the shuttle capabilities and whether or not we can satisfy those requirements. In a very complete iterative process we go through understanding what do they want to do, what is it that we're able to do, and to the degree that it doesn't match up, let's try to better understand how we might be able to force a match there. When you get to the FDRD, you know the vehicle you're going to fly on, you know the size of the crew, you know how much cryogenic oxygen and hydrogen's going to be on board, because that drives how long the mission can be because, of course, that's breathing oxygen for the crew and that's also what we use to generate electrical power for the payload and for the other systems on board the orbiter. You know exactly what the payload configuration is going to be in the payload bay, down to the keel and the trunnion attachments on the side walls of the vehicle. You know probably the serial numbers of the engines you're going to fly. It baselines everything there is that you really need to start out to do the detailed final preparation for the mission, and that baseline can only be changed from that point on by going back to the Program Requirements Change Board and asking permission. So that's the FDRD, and it's really the first milestone at the program level. The other blocks as we follow along there have names which are fairly self-explanatory of what they do and what we're there to do. The Cargo Integration Review highlights and further refines details with the payload that we're going to be carrying for that mission. The Ascent Flight Design is a program-level review because that is understood to be the most dynamic phase of flight. It's the one where we have to tailor the software the most from flight to flight, given any one of a number of variables, not only the payload you're carrying and the weights involved and the load of propellants that you're going to carry on that particular flight. So we bring that to the program level. The FPSR, the Flight Planning and Storage Review, is the one that's near and dear to most crew members' hearts because that usually happens at about the ten-month-or-so month prior to flight and that's just about the time that the crew has just been named and has started working together as a crew. So that's the first one that the crew normally supports; and the Flight Plan and Storage Review, it really summarizes the issues which are most importance to the crew. The flight plan tells everybody, including the crew, what you're going to be doing every second of every mission; and if you can nail down the flight plan and make it answer back to the requirements of the flight, it's a lot easier on the commander to be able to plan his mission and to plan his training for his crew, which is one of the primary jobs of the commander pre-launch. The other important part is stowage. Living on board the space shuttle and working on board the space shuttle has been likened to a camping trip in a closet in that you have to know exactly where everything is so you can get to it in a timely fashion and you also have to get it back in the right place before you come home. And the degree to which you don't know that or you make it more difficult than it has to be, it directly impacts your ability as a crew to get your work done. So you try very hard after you're first named as a crew to get to the Flight Planning and Storage Review and understand the degree to which we have a high level of fidelity in that planning process, because that's your first clue as to how much work you have in front of you in planning the mission, the details of it. The next three blocks really have to do with the same subject, and that is at the Kennedy Space Center what are they going to have to do after that orbiter lands from its previous mission until you launch it on its upcoming flight. The first block, the Integrated Launch Site Requirements Review, is where you hash out what are the actual requirements. You know you've got to be able to get the payload into the payload bay. What are the requirements before and after and leading up to that event? What are the modifications that you want to do on this vehicle? At any given time in the shuttle program, there is usually a list of modifications which are ready to go to be implemented in any given vehicle, and you have to weigh is now the time to try to insert any of that particular modification to bring the improvements that it does either to the capabilities or to the safety level, or do you have to understand that the manifest at its current state is such that work would be better implemented one flow following this flight. So you make those trade-offs and those kinds of determinations at that time. Then the Kennedy Space Center comes back at the Launch Site Flow Review and they tell you their ability to meet those requirements and that they're going to be able to do it and to the degree that there's a mismatch, we hash it out at that meeting. There's one other meeting, the Delta Launch Site Flow Review. By the way, the timing is 60, , plus 15. That is, it's about two months prior to the landing of the orbiter from its previous mission that you really try to nail down the requirements. It's about one month prior to that landing that you do the flow review and have Kennedy come back and tell you if they are going to be able to accomplish it. After the landing from the previous mission has accomplished and they've been able to roll the vehicle into the processing facility, you understand better the condition and any in-flight anomalies which it had during the previous mission, how that might impact what you had planned to do previously. You bring that back to the program at the Delta Launch Site Flow Review and that's where you make any final determinations and judgments on what we are and are not going to do on this particular flow. If necessary, you adjust the launch dates to meet those requirements. So that's the program level review, starting at 16 months prior, to actually up to two weeks after the landing of that orbiter and you start to process the vehicle. This is what's typically referred to as the flight preparation process. The last block that I'll lead into with the asterisk is called Milestone Reviews, and this is going to be where we now tend toward more of a Certification of Flight Readiness flavor for what we're doing. If I could have the next chart, please. I believe I've talked about all this. So if we could press on to the next chart. The next chart, please. Here we go. Here's the wiring diagram to talk about the milestone reviews and the certification of flight readiness that results from this process. The chart flows from the left to the right. On the left-hand side, you have the different projects and elements, each one responsible for a particular major system on the orbiter. On the far right-hand side, you have our flag -- I'll call it our flagship review, the Flight Readiness Review, which typically happens about two weeks prior to launch, where we present all the information to senior NASA management to determine the final readiness for launch; and everybody's required at that point in time to sign up to the Certificate of Flight Readiness. In between is an incremental improvement at each step in our ability and a refinement in our ability to say, yes, we are headed toward the satisfactory Certification of Flight Readiness. Starting at the left on the project level, their major review would be the Element Acceptance Review. That's where the government project manager will accept from the contractor the piece of hardware. Once again, there's a whole hidden set of pre-reviews that led up to the Element Acceptance Review. I've talked to a number of project managers and I think they'll all tell you it would be totally unacceptable for them to be surprised or to hear an issue at the Element Acceptance Review that they did not previously know about. So it's worked in real time, but we do lead up to the EAR for each major component of the vehicle. Then where I've gotten involved in my job as the launch integration manager are in the two double-bordered boxes that you see there. The ET/SRB Mate Review and the Orbiter Rollout Mate Review. Each of those represents a processing milestone that we want to be very careful and we want to be very studious, if you will, before we go through that milestone, without taking a breath and stopping and pausing and making sure we're ready to go do that. I approach it from the point of view of two aspects. First of all, those mate reviews were my opportunity as the integration manager to actually understand the rationale that was going to be brought forward at the Flight Readiness Review for any of the major waivers, hazards, first-time flight items, changes to processes, in-flight anomalies to be considered up to that point in time. It was my opportunity to hear that in a formal forum and to understanding how they were going to present it to the Flight Readiness Review. Now, let me make it immediately clear that, just as it would have been unsatisfactory for a project manager to come to an Element Acceptance Review that did not know everything that he was going to be told, it would be equally unsatisfactory for me as the launch integration manager to come to a mate review and not know the details of everything that was going to be presented and have had a history of having known the development of all those issues over the prior months. Nevertheless, that's the first time we put it all together in one package. ADM. GEHMAN: Let me interrupt. This is where -- I mean, you mentioned this. I just want to be clear about this. In the Element Acceptance Review, these EARs, as well as at these program reviews, previous waivers and waivers that are currently in existence, disposition of old in-flight anomalies would all be brought up, kicked around the table, and if they had been accepted in the past, the acceptance would be re-agreed? COL. HALSELL: Yes, sir. I believe I understand the intent of the question. There is a requirement both at the project level and at the program level for us to fully understand in-flight anomalies as they apply to that particular piece of hardware and the mission that's about to be flown. There's a requirement to review and understand all the waivers that had been issued and, in particular, concentrate on any change of waivers or any new waivers. If it's a waiver which has previously been approved through the program and through the entire system and there is nothing different about it's applicability or this flight as compared to the previous flights, then it's not necessary that it be brought forward again and again and again; but what is absolutely required is that any new waivers or changes to waivers be highlighted at each of these progressive milestones. ADM. GEHMAN: Just from an administrative point of view, if a system over a period of 20 years is operating under 25 waivers -- which, by the way, that's probably not an outlandish number; it might be more than that in some cases -- how does the system deal with the fact that a waiver's starting to accumulate. COL. HALSELL: I am aware during the time that I was at the Cape that the program approached that exact issue at least on a couple of occasions. Just before I took over as the launch integration manager in the summer of 2000, my immediate predecessor, Mr. Bill Gerstenmaier, under Ron Dittemore's direction, had gone through a review of the waivers. The question was: How many are out there? Are they all still valid? How often do we review this situation so that we're not guilty of unknowingly accumulating waivers? To what degree are we confident that we have good rationale for retaining waivers in place? What we found out from that review is that we do have a good process in place. There's an annual review of the waivers to make sure that it is still appropriate, it's still applicable, it's still necessary. Remember, we should probably back up a step and just talk a little bit about how you go through the process of granting a waiver. What you want to do, to the degree that you can't meet the requirements that you have in place, you want to try to change that and satisfy the requirements. So your first goal would be to try to execute some type of design change that allows you to satisfy that requirement. To the degree that that's not possible, then you look at other mitigating factors, if you're able to put warning devices or safety systems in place or a crew or ground work-around procedures in place which mitigate the risks. Those are the kinds of things that have to be part of the acceptance of the residual risk when you do go forward with a waiver. ADM. GEHMAN: Okay. Thank you very much. That answered my question. So the kind of legacy waivers then are reviewed annually or periodically, depending on what the project manager wants as a kind of bring-up. COL. HALSELL: Right. Once again, we concentrate most directly -- in the Flight Readiness Review process and the Certification of Flight Readiness for a particular flight, what you want to know is what's changed from this mission to the previous missions or those waivers which need to be highlighted due to the operational flavor of this particular flight and maybe being different from recent previous missions. You'll make sure that those differences, those deltas, as we call them, that's what you bring forward. The same would be true for the failure modes and effects analyses, the hazards, the program hazards. So there is a family of processes which we sometimes capture in this one word "waiver," but they're all reviewed and all brought forward as required during the Certification of Flight Readiness process to make sure that we're not guilty of missing a waiver rationale that is in need of review prior to that upcoming flight. MR. WALLACE: You said that it would be unusual at an Element Acceptance Review for something to come up that you hadn't heard of previously. I have to say in the weeks learning about the FRR process and even the Launch Readiness Review just done in the days before the launch at the Cape that this is sort of a recurring message, like the work is kind of done before these meetings. I'm curious is it fair to say that these meetings, then, don't get scheduled until the work is done or is it unusual things get stopped at these meetings? Does the meeting become sort of a sign-off formality? COL. HALSELL: I guess the best way to answer your question would be to talk a little bit about my personal experience in this area. When I stopped flying on a shuttle crew for a while and I went down to be the shuttle launch integration manager, I perceived some of the same flavor that you're talking about. That is, the important work was being done and being done exceptionally well -- so well, in fact, that when we got to some of these milestone reviews, it appeared to me that all of the hard issues had been discussed, all of the hard decisions and trade-offs had been made. So I questioned the value to our senior management of these level of reviews; but after being in the job for a longer period of time and after having discussed this situation with a number of my project managers, they had a different point of view. They didn't disagree with the fact that the way we do business is such that most of these problems, not always, but most of them, have been flattened out prior to the formal review, but it's because of the presence of those formal reviews and the fact that you know that senior NASA management, the people that you answer to and the people who are ultimately responsible for the safety of the upcoming mission, 'cause you know they're going to be there to hear that story, it drives all that outstanding work that happens before. So from the point of view of the projects and the elements, they did not want to change or consider any dramatic changes to the forum or to the agenda of any of these reviews because, from their perspective, they were driving the kind of reaction within the system that was healthy and needed. DR. LOGSDON: If I heard what you just said correctly, then what's presented to the senior managers is the situation after things have been smoothed out. How much visibility do the senior managers have to the process of resolving issues prior to the formal reviews? COL. HALSELL: Let me see if I can say it in a clearer fashion. I believe that the senior management within NASA, since the Challenger disaster, serves a critical role in deciding upon the final readiness to go fly safely, and it's our job as the middle-level managers to provide them with the information that they need to make that determination. I believe that the process we have in place works very well to do that. I believe that absolutely if we get to a Flight Readiness Review where there are any outstanding issues or if there are any issues that need to be discussed to the infinite level of detail for that level of management, we do that; and I can recount a number of instances where a Flight Readiness Review which was marching along according to the agenda and there were no particular issues, we would come upon one that required the next hour of discussion. It would require a number of people to stand up ad hoc and discuss their participation and their rationale. The Flight Readiness Review board, as would my board on the orbiter roll-out and the mate reviews, if there was something fuzzy or something that we did not agree with or something that we needed additional clarification, we would delve into those details at that board, up to and including the flagship review, the FRR. The point I was trying to make earlier was it's knowing that you are subject to that level of review and that level of detailed review, if necessary, that drives all the good work leading up to it. DR. RIDE: This may not be quite the right time to ask this question. Maybe it should be further on in your preparation, but you've now mentioned twice that since the Challenger accident, processes have been improved and put in place. I just wonder whether you could elaborate on that and maybe be a little bit specific about changes that you are aware of. There were, of course, FRRs before 51L, PRCBs before 51L, senior management was pretty heavily involved in the key meetings leading up to a launch. I'd just be interested in your assessment of what changes have actually taken place. COL. HALSELL: Thinking back to some of the Challenger findings and recommendations, I believe there were ten major findings and recommendations and then appendices behind that. I know that NASA responded to each and every one of those. The two that come to mind, one that's particularly important to me because it has certainly affected my life, was the thought that we needed to involve the astronaut corps in more of the middle and, if appropriate, later in their career, senior management jobs because bringing that operational expertise over to the managerial side of the house was value added to the entire system. I do know that, for example, immediately after the Challenger accident, a number of astronauts were consciously moved into management positions and we have retained that priority for astronauts as part of their career progression ever since then. I don't know the degree to which astronauts were involved prior to the Challenger, but I know that, after, the answer has been quite heavily and in numerous occasions. I know that another finding from the Challenger commission had to do with the fact that on the specific decision to go ahead and fly, given the new data that was brought forward the night prior to that launch, that information, that discussion, the dissenting opinions and the method of which it was finally decided that we were going to go fly that day, all that was not brought forward to senior NASA management in a timely fashion; and I truly believe that today, given the processes that we have in place -- and you'll hear more about the Mission Management Team later on -- that would not be the case. That issue would have been elevated to the appropriate level, given the same set of circumstances today. DR. RIDE: I guess I was just curious whether you could point to any specific -- and again, this may not be the time -- but any specific parts of the process that have been added or specifically strengthened in the pre-launch process. COL. HALSELL: I guess I can speak to the strengths of the processes that we have in place. With regard to the details of comparison how it was pre-Challenger, which was prior to my participation, I probably would not be the right person to ask; but when I get to the part about the Mission Management Team and the process that's in place, I would invite anybody who is knowledgeable about being able to compare that specifically to what we did pre-Challenger to help me out there. GEN. HESS: Colonel, before we get too much further in your briefing, which might be in question, I was curious about providing some balance in the discussion with regards to the line responsibilities to the requirements meetings and these various reviews and how that is balanced by the S&MA organization and recalling the Rogers Commission saying you needed an independent safety process. So if you could help us out at these various stages and give us some idea about how safety figures in and whether or not they can actually overturn one of these meetings because of their degree of questioning over any particular portion of the mission as it's going. COL. HALSELL: Let me answer the last element of your question first, and the answer is absolutely yes. On each of the reviews that I've participated in, whether it be the orbiter roll-out review or the mate review, the safety community is represented through several different channels. Also, the pre-launch Mission Management Team review at O minus 2 -- that's launch day minus 2 two days -- and then at the Flight Readiness Review, Safety is always there. They're always represented and they are always polled and they always expected to come forward with a dissenting opinion which would cause everything to stop at that point in time and we not progress to the next review on the right side of that chart until we had it hashed out. So that's the answer I want you to hear is that Safety absolutely has not only the ability but the requirement to step forward if they believe that the engineering community is headed down a wrong path. I believe that's the essential element of one of the strengths of the processes that we put in place. That is, that, in my opinion, a large part of your safety that's built into the system is accomplished through the strength and the viability of your engineering community and their in-house safety work that they do in line. But it's also important -- and I know that Ron Dittemore has always felt very strongly about this -- it's also important that we have an independent over-the-shoulder assessment of how we're doing from the safety community also. And the important aspect that we've always worked hard on is making sure that as we do our job in line, we have that independent assessment looking over our shoulder and then the fact that they are staffed, have the resources, and empowered to give that independent look at what we're doing. That's the fundamental strength, I believe, in the process that we have in place. ADM. GEHMAN: Colonel Halsell, we're using the term "waiver." You already said this. I just want to clear it up. We're using this term "waiver" kind of loosely here because it really characterizes a number of administrative steps that are taken to account for processes. Can you mention what some of those other ones are called? COL. HALSELL: Yes, sir. Some of the other categories that we talk -- for example, hazards. Hazards are a top-down look. You start with a fairly limited number of ways that you can lose a vehicle or crew and then as you drill down deeper and deeper and you spread out farther and farther, you understand the more detailed failures that could cause that hazard to be recognized. The shuttle program is designed to avoid these hazards and, to the degree we are not able to do that, then we try to control them. You control them by looking at your design and implementing changes, if possible, or the safety controls or warning devices or crew operational procedure work-arounds that I talked about earlier. ADM. GEHMAN: Is that what you referred to as a FMEA? COL. HALSELL: Well, a FMEA CIL is actually a different process. It's a bottom up. It's where you talk about, all right, what if that component of that box failed? Then at the box level, what if this avionics box fails or this component within my auxillary power unit hydraulic system fails? What's the worst thing that could happen to me as a result of that? We have requirements within the system, as I explaining at the beginning of the discussion, with regard to our willingness to expose ourselves to risk. We always want to be fail-safe. We desire to be fail-operational. The degree to which we're not able to meet -- and you also use a risk matrix approach, if you will, in analyzing some of those risks associated with the different failures. Basically it boils down to looking at what is the probability of an occurrence of a particular failure and what are the consequences if that happens. Depending upon where you fall in that risk matrix determines whether it's unacceptable, in which case you don't fly and you make a decision to go fix it -- and I can give you examples of those kinds of cases -- or if it's an accepted risk because you believe that the mitigations that you have in place make the combination of probability and consequences a safe situation for you to go fly in. Then a totally controlled risk is where you don't believe there is any significant risk that you're being exposed to. ADM. GEHMAN: If we took a case like the cause celebre of the day, foam hitting the orbiter, if during the course of the years that foam shedding and foam hitting the orbiter had been previously waived and had previously been disposed of, it's likely it would not even have come up at the ET review. Let me rephrase that. That's a question, not a statement. COL. HALSELL: Yeah. And I believe my correct answer to your question is that I don't believe that to be true. We'll use that as an example, if we want to pull on this thread a little bit. I think it's well known that we did liberate a piece of foam on STS 112; and the process by which we went through understanding what had happened, how that related to our previously accepted hazards and FMEA CILs and what was the appropriate course of action from that point on all followed the processes that we had in place to try to ensure that the right decisions and the right trade-offs and risks got made. For example, in the in-flight anomaly situation for STS 112, that did come to a Program Requirements Change Board. It was decided there that an in-flight anomaly designation was not required for this particular item because the previously accepted and documented hazards -- and if I remember correctly, there were two integrated hazards which were violated or which were called into question by this particular instance -- two of them dealing with the external tank liberating foam and creating a hazard to some other vehicle component -- there was nothing about that particular instance which invalidated the rationale for the previously accepted risk. In other words, we didn't move up and to the right on the risk matrix, according to what we knew at that point that time. So the action that was levied at that Program Requirements Change Board was to the external tank project to go back and fully understand what had happened, why it had happened, and what we were going to do to keep it from happening in the future. Also another action was levied to bring that item forward at the Flight Readiness Review to make sure it was discussed prior to STS 113. So using that as my example, I would say that that's an example of how the process worked properly and the item was brought forward to the Flight Readiness Review and it was discussed at some considerable length there. DR. RIDE: How would that have been different if it had been classified as an in-flight anomaly after 112? What would have been different in the disposition process? COL. HALSELL: Nothing. In the sense that whether it's designated in-flight anomaly or not, the important item is that two PRCB directives were issued at that time which directed the project to go back, analyze the problem, find out what it is, and fix it. Another action was issued to make sure this was brought forward to the Flight Readiness Review. So whether it's designated an in-flight anomaly or not, the answer is it would have made no difference. Now, let me jump ahead and make sure that I'm not guilty of not answering the question you meant to ask, which is, if we had designated at the highest level, which is in-flight anomaly with constraint to next launch, then it would have been immediately an issue which had to be not only fully understood but resolved either with an engineering design change or an appropriate rationale for flight and formally documented. So on this particular case, I would maintain that that process was worked, because we did discuss this issue at the STS 113 Flight Readiness Review at some length. The process of making sure we felt comfortable and safe and that we understood the risks and the hazards and that there were no significant changes from those that had been accepted in the past, all that was done, despite the classification that we came forward with at the PRCB. MR. WALLACE: If I could follow up. I understand from reading some of the PRACA documents that all PRACA reportable items must be dispositioned in some way -- I mean, prior to the next. Is that a fair statement? COL. HALSELL: Yes, it is. However, there is sub-documentation that gives you guidance by which projects are allowed to enter into interim disposition as opposed to disposition prior to the very next flight. And it was the consideration of that particular set of guidance, of rules, along with what we thought was an understanding of no significant increase of risks due to the liberation of STS 112, that led the PRCB to decide that the appropriate way to deal with that particular issue was to issue the directive for the external tank project to come back and find it and fix it and tell us what they had done and also discuss it prior to the Flight Readiness Review. In general, yes, all problem resolution reporting and corrective action items have to be dealt with. The level at which they get dealt with depends upon the criticality, Criticality 1 being the most significant and requiring the highest level of managerial insight and concurrence with. On the other end of the spectrum would be Criticality 3, which means you have no risk of loss of vehicle or crew. Those can sometimes, under the guide rules that we have written down, be dealt with at the project level and with different combinations in between going to different levels of management. I would hasten to add that, as a project manager or as a program person, you don't have the right to decide, on any given day, what level it's going to go to. That's all been decided for you, and it's documented for us in our processes. MR. WALLACE: So this item which was a PRACA reportable item but not an in-flight anomaly on 12, there was an interim disposition? COL. HALSELL: Yes. MR. WALLACE: Which then didn't include any hardware changes -- it wasn't an assignment to -- COL. HALSELL: We can read the exact directive; but paraphrasing as I remember, it was: "ET Project, you've got until the 5th of December -- and I think that date was later extended due to some conflicts of scheduling -- but you've got until the th of December to go find out exactly what happened, reinforce for us what you're telling us today, which is you have no reason to believe that it's a generic issue and that we're at any increased risk on the upcoming flights of suffering this problem. We would like your options for engineering design changes which could be implemented to completely alleviate this problem in the future. Come back and report to us what your options are and what your recommended plan is." MR. WALLACE: Could you tell us about the decision-making, I guess it was in the post-112 PRCB, the roles of different elements in the decision-making as regards the classification, in-flight anomaly or not, and the decision to go with an interim deposition, particularly the external tank element and the S&MA office, if could you speak to that. COL. HALSELL: I'm trying to think, Mr. Wallace. What additional information or what avenue are you trying to get me to talk about specifically that I haven't talked about already? MR. WALLACE: Just really focus on who makes the call on that, on the in-flight anomaly decision and on the interim disposition items. COL. HALSELL: You're doing a good job of doing my presentation for me -- and that's fine. That's good. Let me. If I can go to the final two slides, if I remember, in the presentation, prior to the backup. Let's cover the two in-flight anomaly pages. After every flight, or as you're doing the flight, every element, every project, including Mission Operations Directorate, which Bob will have an opportunity to talk about here in a moment, they're compiling their list of things which have happened during this flight. Sometimes you hear it called the funnies list or the action log. It goes by a number of names depending upon which element or project you're talking to. I'll use the name "funnies list." That's everything that happened that was worthy of attention by somebody. In general, that entire list, all the problems, all the elements, all of their funnies get brought to the Program Requirements Change Board. Usually it's the first one following the landing of that vehicle. Sometimes it goes to the second PRCB. The program documentation says we need to do it no later than two weeks after landing, is our general goal. It's a fairly long and detailed PRCB agenda item where you go through each and every problem that you experience, all the engineering information that you know that might have caused it, and the elements first blush on where we need to go from here. As part of that and as we go through each and every one of those items, it's a PRACA reportable item. You never have the option of saying, well, thank you very much but I don't think that's worthy of my attention. Everything gets dispositioned one way or the other, and part of the process that everybody is focusing on appropriately in this discussion is in-flight anomaly or not. What you see before you are the listing of rules by which the funnies can get elevated to an in-flight anomaly. Just to go through them briefly, if it's a Criticality 1 or 2 -- meaning that we threaten the loss of vehicle or the crew, Criticality 1, and Criticality 2 meaning we threaten loss of a normal nominal mission, that's worthy of in-flight anomaly consideration. If it's software, either orbiter flight software or the space shuttle main engines, it could cause Mission Operations Directorate -- and Bob can probably give us examples of these kind of situations where we got the nominal mission accomplished but they had to work extra hard and had to do a lot of work-arounds on orbit to make that happen -- then we don't want that to have to happen again. So we deal with that as an in-flight anomaly. If it caused or if it could have caused a countdown hold or a launch scrub or a launch abort, then we want to deal with that. If it could have affected safety or mission success or caused significant impact on resources, logistics, or schedules for the future, or if it's any anomaly that the designated responsible design element wants to make an in-flight anomaly, they have the final word. So that's a list of things that we use as criteria for consideration as in-flight anomalies. If I could have the next slide, please. As far as interim deposition is concerned, these are some of the items by which it was appropriate for us to give the elements more time to deal with these issues and not call them constraints to the very next flight. Let me run through those. Remember, it's one of the following criteria: If it's not applicable to the flight -- in other words, whatever broke last time, you're not flying next time, that's obvious; if the problem condition is clearly screened during pre-flight checkout or special tests and you know you're not subject to that same problem; if the problem is time/age/cycle related and the flight units will accumulate less than 50 percent of the critical parameters by the end of the upcoming flight; if there's no indication that this is a generic problem or if you have no overall safety-of-flight concern; if the problem is applicable to flights, however, the PRCB agrees that we have sufficient evidence that the system can be flown safely with acceptable risk, then those are the kinds of circumstances under which we would go to an interim disposition. And it's my belief that it was the consideration of these type of issues which led to the determination that the external tank foam, using that as an example, would be an appropriate issue for us to talk about completely at the upcoming FRR but to give the project additional time to come forward with their corrective action. MR. HUBBARD: I'd like to go a little bit to the hand-off between the end of one mission and the beginning of another. You just characterized what you do post-launch. Now, let's go pre-launch to the next mission. What is the process by which the collection of things that have happened over the various missions get put into a data base or some kind of a memory bank, other than just individuals around the table so that, as the missions go forward one after the other, you build up a sense of trends? You know, maybe there's nothing on one specific flight, but maybe there's an accumulation. How does that get brought to the attention of management during the review process? COL. HALSELL: I believe the answer to your question is PCAS, which stands for Program Compliance Assurance System. Lately the new word is web PCAS in the sense that its been upgraded to a web-based system, and previously it had been a mainframe-hosted computer system. Web PCAS is a web-based system which allows any person associated with the program at any level, including senior management all the way down, to access all the sub-data bases. PRACA's been -- the problem resolution reporting and corrective action system, that's one of the sub-data bases which is part of PCAS, for example. The waivers list. The in-flight anomalies list. The FMEA CILs. All of these data bases -- and we could probably go on for quite some period of time to have an exhaustive list -- are part of the web PCAS which the engineering community and the safety community use equally in this type of trend analysis and in what we characterize as the paper close-out that has to happen before we go fly again. Before we fly, we have to be 0 percent sure that we have our requirements and our closed-loop accounting system has sufficiently -- you can't launch if you simply know nobody's elevated a problem. You have to have the reassurance that people have looked and that they have closed out all of the open paper, and it's only upon that positive affirmation that you can go fly. MR. HUBBARD: So just to follow this one step further. This data base is available. Is there anybody who is charged with actually looking at it and as you go around the FRR and these other reviews saying, wait a minute, to take our favorite topic, I see a trend in foam-shedding or something like that? COL. HALSELL: Yes, sir, and there are two somebodies. Every project and element -- and you'll see the participation in the Flight Readiness Review -- every project and element associated with the program has to say that verbally at the Flight Readiness Review. They are signing for that when they sign the Certificate of Flight Readiness that, yes, we have looked at this and we know we have closed out all these issues; and the independent assessment that we were talking about earlier, that's an important part of their function in ensuring safety is they look over our shoulder and they make sure that every project and every element has closed out those issues appropriately also. ADM. GEHMAN: Could I ask you to go back one viewgraph here. I don't want to talk about STS 107 specifically. We're talking generic processes here, but I would like to talk about foam-shedding as a generic process. So if you can go back one viewgraph, please, to the in-flight anomalies, the IFA. Thank you. Okay. So as I understand it -- and I don't know whether this viewgraph comes from NASA regulations or procedures or where it comes from, but I'm going to assume it's accurate for right now -- we, of course, will check that out -- it says there that any one of the following criteria makes it an IFA. I assume that damage to TPS, since it's Crit 1, that Item A there, any problem that affects a Crit 1 system which is damaging TPS, we've got ourselves an IFA. COL. HALSELL: Yes, sir. I mean, reading No. A, that's what it says; and I would once again draw your attention to the second page which we've already covered, which gave further guidance which would allow an interim disposition. ADM. GEHMAN: Now, I want to go to the second page. Once again, I'm not talking about the FRR of STS 107. We're going to go into that in some detail. I'm using this as a generic case. It looks to me like something hitting the thermal protection system or damage to the thermal protection system is a Crit 1 system and therefore anything that hits the TPS ought to be an IFA, looks to me, just using this score card. And if we look through the disposition here, it says that interim disposition is acceptable or a final closure is required if you meet any one of the following criteria. So I look at A, problems not applicable to the flight we're talking about -- that doesn't apply. A problem condition is clearly screened pre-flight -- that doesn't apply because you can't tell what piece of foam is going to fall off. C doesn't apply because it's not age related. D, I would say, doesn't apply because it's a generic problem and can happen anytime and anyplace else. Then we get down to E: There is no safety-of-flight concern. Now, can you tell me how -- or even the last one: The board agrees that sufficient evidence exists that the system can be flown safely. How in the world does the system determine that there's no safety of flight? Do you know what processes there are involved or is it judgment or... COL. HALSELL: I know you say we're not going to discuss and this is not STS 107 related, but it is ET foam related. So continuing with that as our example, as I remember, the particular presentation at that PRCB, the nature of the rationale that was presented in that forum was that the external tank had gone back even at that point in time before they had responded to the following action and they had vigorously tried to understand did we do something different with the tank where we had this problem as compared to all the other tanks which had flown successfully. What came out of that was they felt comfortable that there was no new and generic issue that they could identify, either with changes or weaknesses in their processes in applying the foam or manufacturing or in the vendor that provides the raw material. They had already gone back and looked at all of that and they felt comfortable at that point in time that they had no generic issue that indicted follow-on future tanks that we were going to go fly. Furthermore, I do not know for a fact that it was presented in that form but I do know that as part of the Boeing transport mechanism there was no elevated level of concern that anything liberated from that location would have impacted the orbiter. What all this added up to was the conclusion that we had not moved up and to the right on the risk matrix with respect to the previously accepted hazard, the two hazards that had been accepted and which we had flown for much of the life of the program, I believe, since STS 27. ADM. GEHMAN: Thank you for that. To follow up on Mr. Wallace's question, is it the PRCB that would make that decision that there is no safety of flight or -- I mean, it wouldn't wait for an FRR; you would have settled this some other way, I presume. COL. HALSELL: It isn't the Program Requirements Change Board, that the program manager has the ultimate responsibility for determining what are we going to classify as an IFA, what are we going to classify as an IFA with constraint, and which are we going to classify as an interim disposition with an action assigned to come back at a later point in time. But also it's important to understand that the Flight Readiness Review, upon review of any of those actions, certainly has the ability to upgrade any item that they so deem necessary. ADM. GEHMAN: Absolutely. DR. LOGSDON: I am going to ask a question about STS 107. If the mission had been successfully completed, would the foam shedding have been classified as an in-flight anomaly and, if so, by what criteria, since there was an analysis that said it was not a safety-of-flight issue. It was counter-factual, unfortunately. COL. HALSELL: I want to make sure I answer exactly the question that you're asking, and it's in the context that we have had the foam liberation on STS 112. ADM. GEHMAN: No, what he's saying is Columbia gets struck by foam just like she did but she returns safely. COL. HALSELL: Yes. Absolutely. And given that we have now had a second occurrence -- DR. LOGSDON: Go back to the prior slide. COL. HALSELL: Before you do, just remember "D" there about the generic problem. At that point in time, I have absolutely no doubt that following the STS 112 incident and it happens again on 7, what you now have on your hands is a major issue that has to be dealt with before we consider even rolling out the next vehicle, much less flying the next vehicle. MR. WALLACE: And the fact that on the 7 it struck the orbiter, does this even make it way more clear that this would rise to the level of an IFA? COL. HALSELL: Especially given that the Boeing transport analysis seemed to indicate that we were not at severe risk of having a strike against the orbiter from a piece of foam liberated in this area. Now, to be complete and fair -- and I know you know this -- that same transport analysis also indicated that there were weaknesses in the program that was being used to do this analysis. Perhaps most specifically, they made the assumption that you were dealing with a non-lifting something and that as soon as you implied some lift in a direction, then that would have to undergo further additional analysis that took that into account. ADM. GEHMAN: Why don't we let him move on here. GEN. DEAL: Well, I'll go ahead and ask you an opinion question here, Jim, a little bit. It's based not just on your extensive experience in the shuttle but also your flight test experience. If 1 out of every 25 flights you're flying a test development vehicle and it drops a panel forward of the intake, you know, I would think you would be a little bit concerned. We talked to some test pilots that say the deserts around Edwards are littered with panels out there, but, you know, I equate foam falling off of a bipod and hitting some part down below that's critical to the flight as being something forward of a jet intake. Can you give us any perspective about if we showed the right level of concern with four previous bipod ramp incidents where the foam broke off as compared to what type of precedents we put on it. COL. HALSELL: I understand the context of the question you're asking me. As a test pilot and somebody involved in the job of acquiring the data with which a vehicle that's going to be flown for hundreds of thousands of hours over the fleet and making sure that we vet out all those issues while we're in the test phase, as opposed to in the operational phase, trying to transfer that experience to what we're dealing with here. One of the limitations that we've had over the entire life of the shuttle program is that we've never had the opportunity to accumulate the number of flights and the number of flight hours and the number of occurrences of any particular item to be able to apply the same statistical rigor that we're able to do in flight tests, for example, where you do quickly accumulate that kind of experience. I think trying to draw that analogy or that comparison might be an error on my part. So I would ask that I not be asked to do that because I don't feel comfortable doing so. I will take what I think is the intent of your question, and that is at the point in time when STS 112 occurred, we had not had a loss of ramp foam, if I remember correctly, since approximately STS 50. There might have been some interim problems with ramp foam, but nothing of that size and significance. Following STS 50, they had changed some of the procedures and some of the foams; and we thought that had been an improvement in our processes and in our materials. So when STS 112 happened, whether it was appropriate or not, I think there was a consideration that this was a new occurrence, given a new baseline, and trying to statistically infer that what had happened prior to those changes were applicable to our current configuration was not appropriate. I'm sure that that consideration will be something that the investigation board will feel charged to draw an opinion on. GEN. DEAL: I've got two other questions. Since we're controlling your briefing for you, if we can go back to Slide 10, I've got a question for you because we haven't covered that one yet. We bypassed it. When I look at the FRR, Jim, I see a lot of people in there. Some of them are former astronauts. Is the mission commander involved in this? Are the current astronaut corps involved in the FRR? COL. HALSELL: The Flight Readiness Review, the flight crew is represented to the board or the Flight Readiness Review through several different avenues. The center director for the Johnson Space Center, the astronauts are hired and work for that person. So he represents their interests. The manager of the space shuttle program -- GEN. DEAL: On the three that you commanded, did you attend the FRR? Were you a part of it at all? COL. HALSELL: No, I did not; and, furthermore, I think that that's the right thing to do because sitting right behind the board, not at the board table, as the commander of a shuttle mission, I have my direct and immediate two people I consider to be my reps to the board. That is the chief astronaut, that's currently Kent Rominger; and the director of flight crew operations, currently Bob Cabana. Those two individuals, in my opinion represent the flight crew, the flight crew interests, the flight crew point of view, and that's who I want to be there and to concur with any issues having to do with the Flight Readiness Review. Now, I think there's a page of presenters here; and I forget if it's forward or backward. But very close to here is going to be the agenda. There we go. You should see flight crew and the left-side halfway down, the flight crew operations director will make his presentation to the Flight Readiness Review board as to the readiness of the flight crew to press forward into launch countdown. At that point in time he's certifying that the crew has been fully trained, is ready to go fly, they have all the procedures, they've been trained on all the procedures, they have all the equipment and training on how to use it to accomplish the mission. Bob Cabana, the FCOD director, doesn't just stand up and say that. In preparation for the Flight Readiness Review, he has a pre-FRR at which the commander of the mission does attend; and it's at that meeting here at the Johnson Space Center approximately three to four days prior to the FRR. It's the face-to-face meeting where the FCOD director queries the crew commander and asks him: Are you ready to go fly this mission? Do you have any concerns? Do you have any issues? So I feel 100 percent justified in saying that even though the flight crew is not physically present at the FRR, they are 100 percent represented in terms of their ability to make it known to anybody and everybody if they have a question. I guess I feel like I know something in this particular area that I would like to express. There are about 100 meetings that you don't want the flight crew to go at. Because at this point in time in their training, two weeks prior to launch, that's when their highest task loading is. That's what they're trying their hardest to -- it's actually now in the preceding two or three months they're trying to congeal together as a crew, ingrate all the procedures, all the issues, and at this point in time they're typically involved in the terminal countdown demonstration test where they go to Kennedy Space Center and participate in a full dress rehearsal where from the time you wake up that morning until you do the simulated emergency egress out of the vehicle, every step from waking up, suiting up, going out to the briefings, going out to the pad, getting strapped into the vehicle, going through all the procedures of the last couple of hours of the countdown, that's what you're concentrating on. And I would maintain that as important as it is to make sure that there's a chain of communication from the command to senior NASA management, it's also important that we don't overburden them with an unnecessary requirement to be at certain meetings. We just need to make sure they have that communication path; and I believe certainly for all our reviews, including FRR, we do. ADM. GEHMAN: Go ahead. GEN. DEAL: I've got one more follow-up, but I can wait. COL. HALSELL: Did I miss a question? ADM. GEHMAN: No. Go ahead. COL. HALSELL: With the presentation? I've kind of forgotten where I was. ADM. GEHMAN: Page 6. COL. HALSELL: Okay. Thank you, sir. Let's see we were talking -- the vehicle preparations. Element Acceptance Reviews. And I think I got through the external tank mate reviews. And we got taken down what I -- I said there were two things that as the launch integration manager I tried to concentrate on on the mate reviews. The one we covered in a lot of detail. I called it the paperwork, but it is the close-loop accounting system to make sure that we have positive affirmation, that we have met all the requirements, that the rationale for the waivers that we need to go fly with are in place and still valid. The other part I'll call the practical side. As the launch integration manager, I did not ever want to be guilty of getting caught having gone through a significant milestone such as mating the external tank to the solid rocket boosters or, later, rolling the orbiter out of its protected processing facility and bringing it over to the Vehicle Assembly Building, going vertical and mating it and then finding out that there is something not right, something that I should have known about at the mate review or prior that, in hindsight, would have stopped me from going through that milestone. After you mate the orbiter, for example, you don't have nearly the access that you do in the orbiter processing facility. So there was a practical side to those mate reviews that it was important to make sure we had full understanding of, also. Next slide, please. This slide probably does a better job than I did verbally of answering a question earlier of is there a process by which all the waivers, all the FMEA CILs, all the open hazards, any upgrades in hazards or FMEA CILs, that it's all brought forward, what is that closed-loop accounting process that we make sure we're ready to press forward to the next level of readiness. This slide gives you that, and I think we've touched upon some of the important elements of that. Next slide, please. Now we're talking about Flight Readiness Review, which I think has been done. Let me see if there's anything on this chart that we haven't really talked about. I think the important thing to understand is that the Flight Readiness Review exists at its core for the associate administrator of the Office of Space Flight, Mr. Bill Readdy now, to make a final determination if he feels comfortable that we have done everything that we said we would in our requirements to get ready to go fly safely. Next slide please. This slide should look very similar to the one that I presented two slides ago because it says basically the same thing. We review all the open issues, make sure that our baseline configuration, what we're flying is what we said we were going to go fly and, if it doesn't, that we understand why and that we agree with that. Any significant unresolved problems or resolved problems since the last review and the flight anomalies, any open items on constraints, any and all new waivers and any open actions from the Flight Readiness Review or any of the element reviews that led up to that have to be closed out at this meeting. At the formal end of Flight Readiness Review -- could I have the next chart please. I'll continue my thought in just a moment. Here is the participation of the board. What I might have in the backup charts but, if I don't, I want to make it clear to you, that this is not just a table with these people. It is, rather, a table in the center of a very large room with these people surrounded by literally hundreds of other people. Every project, every mid-level and lower-level manager of each project is represented there, each of the contractors, from the CEO down through every individual that he or she thinks is necessary to provide the necessary support. Literally a couple of hundred people at least are attending these meetings and are right there in the same room. Next slide, please. Some of the logistics are talked about here. We try to hold this review a couple of weeks prior because that's soon enough so that if we identified any issues at that point in time that need to be dealt with, we have some chance of still making a launch date after having satisfactorily resolved those issues. You don't want to do it much earlier than that, though, because you're reviewing a flight for which issues and problems are going to arise in the interim period of time. So that seems to be the right middle ground. We talked about how all the NASA and contractor personnel are there. One important aspect is that we insist that the whole world of the space shuttle program travel to the Kennedy Space Center and be there in person. You do not participate in the Flight Readiness Review by telecon. You will be there and, if you can't, your designated alternate will be there. It's that face-to-face conversation, face-to-face interaction, that allows you to gain so much more information than you can from a telecon and a voice transmitted to you over the telephone. So the face-to-face nature, I think, is something that's important. Also not only do we have minutes but we audio- and video-record the proceedings. I know, for example, in answer to Dr. Ride's previous question, that's one thing in particular I remember was implemented post Challenger that we hadn't done such a good job of previously. Maybe we had been as good at analyzing some of our issues, but the documentation of the way we resolved those issues wasn't as stellar as we would have liked. We made sure that problem was fixed, hopefully, after Challenger. MR. HUBBARD: This is a little bit of a subjective question, but let me start off with just a fact or two. You participated in FRRs as the manager of launch integration, and what you described is a big show. I mean, it's a big deal and it's a big room and a lot of people. Somebody once said if you have more than five people at a table, it's not a meeting; it's a conference. So you've got, as you said, a couple of hundred people, more than a hundred people in the room. What do you feel like when you're in an FRR? What do you think the tone is? You know, people have their antennae quivering, looking for issues? Do they feel like their working their way through a series of boxes? How do you feel when you're going through an FRR? COL. HALSELL: I feel like it is the culmination of a very, very long and involved process. I feel like when we're there in that room, we are putting the important final touches on the work of thousands of people. It is thousands of people. Tens of thousands of people. That filters up at the engineering and manufacturing level, up through the element processes and reviews and the element project managers to what I'll call the mid-level to upper-level management that I participated in in my reviews as the launch integration manager. But it certainly wasn't just me. There are a lot of other mid-level managers doing the same thing in their areas of responsibility. And I feel like the Flight Readiness Review is that flagship review at which we have that last and final opportunity to present our story to senior NASA management. And we know that they've been made aware in an interim basis on everything that we've been doing. But I feel that at the table at the FRR board you have the representatives of the right organizations to lend that final not only senior managerial level but that experience viewpoint and common sense viewpoint and asking the straightforward simple questions: Have you done this? Have you accomplished that? Why do you feel comfortable that your assumptions that you made here allow you to make the conclusions that you're presenting to us? I feel that that's the level of inquiry that we get at the Flight Readiness Review, especially on issues that require that at that point in time. So I feel like it is an appropriate and exhaustive review that culminates an appropriate and exhaustive process. MR. HUBBARD: Just one follow-up on that. People, in general, can feel very comfortable saying things one on one, maybe even in a group of five or ten. I don't know if your average engineer -- and, of course, this is a group of senior managers -- but do you think people feel comfortable raising an issue in a room with a hundred people? COL. HALSELL: I know that in this particular forum there's absolutely no hesitation to raise your hand, even if you're sitting with your back up against the back wall, against the wall of the building -- and it happens every FRR. And I would simply volunteer to bring forward transcripts and also recordings to back up what I'm telling you. It would be highly uncommon for somebody not to interrupt a presenter in the middle of their presentation and say, "Well, now, wait a minute. How can you say that when we had something else happen two years ago which now seems associated. What do you think about that?" At some points in time, as the secretariate, if you will, of this particular presentation, my issue has not been with getting full and free participation but just making sure I get it documented. I've got to stop people. I've got to say, "Please come forward. Make your way to the microphone. We need to get this recorded. We need to understand what you're trying to tell us." So my issue has been just to make sure that those types of input are recorded and documented properly. So I do feel that the Flight Readiness Review is a full and open forum. DR. LOGSDON: If there is that kind of lively interaction at the FRR -- and this is really a question asked out of literal ignorance -- have there been FRRs that have resulted in a decision that the mission was not ready to fly? COL. HALSELL: Yes, sir. We have a way and we have a process to document that. It's called the Exception to the Certificate of Flight Readiness. Next slide, please. I'm trying to see if I have it up here. Next slide, please. Okay. We'll stop right there. What happens at the end of the Flight Readiness Review is that after all the elements have presented, the chair, Mr. Readdy, will typically ask an all-encompassing question. He'll scan the room, try to make eye contact with everybody and say, "Is there anybody in this room who has any information that has not been brought forward that is relevant to making a decision as to flight readiness?" It is rare at that point in time that anybody raises their hand because they should have done it -- and they do do it -- during the element's previous presentation. Nevertheless, Mr. Readdy makes sure he gives that last and final opportunity for anybody to raise a hand and say, "Yeah, there's something here that we haven't talked about yet." Also during the course of the presentation, prior to this point in time, the elements can take an exception to their Certificate of Flight Readiness, which is basically a way of saying: I certify that I did everything that's required by 8117, also the appendix to 8117, which is my element-specific requirements that I'm signing up to, and also the preamble to 8117 which applies to everybody equally. I am signing up that I did everything and I've closed up all the open issues in a closed-loop accounting fashion with the exception of this one following issue; and that's the Exception to the Certificate of Flight Readiness. A last thing we do at the Flight Readiness Review is that Mr. Readdy will poll his board members and contractors and they will have the opportunity to say verbally if they certify to flight readiness. Anybody who has taken an exception to flight readiness will, in addition, at that point in time, verbalize that exception, say something to the nature of, "With the exception of issue of working with shuttle main engine thermocouples" -- I'll just use that as an example -- "we certify that we're ready to go fly the next flight and, furthermore, we will not allow the launch to proceed until we clear this exception to the COFR." You're kind of a good lead-in to the pre-launch MMT because that's going to be the venue at which we clear the exceptions to the Certificate of Flight Readiness, if you'd like me to continue on into that at this time. DR. LOGSDON: As you do that, can you give me a sense of how often you get to a pre-launch MMT with significant open items? COL. HALSELL: Exceptions? I would say that -- I'm going to guess. We can go back and get the exact percentage over the last couple of years, but it is not unusual, somewhere between 25 and 50 percent of the time, I would guess, that at least one exception to the Certificate of Flight Readiness is presented, and it's always presented with the conclusion of Mr. Readdy, "We think we can or cannot clear this exception in time to make the launch date that you're considering and therefore we do or do not recommend that you press forward toward that currently suggested launch date." At that point after the flight readiness poll and everybody's had a chance to say their piece -- and this might play in a little bit to the question that Mr. Wallace had -- it is tradition that Mr. Readdy adjourn to another smaller room with only invited participants. Usually that's going to be the Flight Readiness Review Board, the prime contractor CEOs, the launch director, the manager for launch integration, and a few other selected folks. In that smaller forum, Mr. Readdy makes it clear that if there's anybody who for whatever reason -- and I can't really understand why -- but if there's anybody who wants to say anything there in that smaller forum that they were not willing to come up with in the larger forum, now's the time and place to do that, before we set a launch date. And it is in addition to that information that's made available to the associate administrator at that time that he considers before he presses forward with setting the launch date or not. We can and we do set launch dates with exceptions to the Certificate of Flight Readiness still pending, but only if he has firm understanding and recommendations that we're going to be able to clear them prior to that launch date. If you like, I'll press forward with the next couple of slides. So we've finished the Flight Ready Review process. The members of the board have been polled. We've adjourned. The associate administrator has adjourned and had his opportunity to hear anybody in private and also to decide if he wants to set the launch date. For the purposes of this illustration, we'll say the launch date was set and that we do have some actions and an Exception to the Certificate of Flight Readiness that have to be accepted prior to going to fly. Let's go ahead now to two days prior to launch. Remember, the whole world came to the Kennedy Space Center for the Flight Readiness Review. They now go away and do their business. Two days prior to launch, we require once again that everybody come back to the Kennedy Space Center. We do it two days prior to launch because we want everybody to have a chance to get back, get in place in plenty of time to set their other job duties aside and to concentrate only on the next safe and successful launch. Two days prior to launch, we convene the Mission Management Team. The Mission Management Team -- and I believe if we could go to the next slide, please -- I was thinking that I had a slide that showed the composition of the Mission Management Team. Basically if you go back to the FRR agenda slide, remember all the participants, all the people who participated in presenting the information to the Flight Readiness Review associate administrator, those organizations and their leaders now become the launch integration manager's mission management team. It's totally appropriate to think that we've not had our review by the very senior level of NASA management and they are now handing off to the mid-level management, with their supervision, the job of launching this vehicle safely within the constraints and within the rules that have been set aside for us to work with them.  Columbia Accident Investigation Board Hearing So that Mission Management Team convenes and we go through basically the same agenda that we did for the Flight Readiness Review. Every element, every project gets the opportunity to present any interim issues, anything that has arisen since the Flight Readiness Review. If there are any exceptions to the Certificate of Flight Readiness, the full and complete rationale for that is presented there to the same level of rigor that it would have been presented in the Flight Readiness Review. As the launch integration manager chairing that pre-launch MMT, I felt it was important that I get input verbally and visually and in public from the program manager and from the associate administrator at the MMT that they concurred on that FRR COFR exception. In other words, it wasn't just the middle managers now clearing something that previously wasn't good enough for the senior managers to go with. At the end of that MMT, we, once again, poll all the participants to make sure that they are "go" to press forward with the countdown. From that point on, the Mission Management Team is activated. I know where each of them is. I can convene a meeting in literally an hour's notice if I need to during the launch countdown. The next time we convene will be formally three hours prior to launch, in the Launch Control Center. If I can have the very last slide in the whole package, I believe it's a picture of the Launch Control Center. As he's scrolling forward -- at three hours prior to launch, the Mission Management Team will convene in this room that you see. Next slide, please. Here's another view of it. Up and in the dark to the upper left is where the Mission Management Team resides. The larger room is the Launch Control Team and the Launch Control Center under the direction of the launch director, who stands just about underneath that American flag in the center of the room. It can help you to understand the relationships here as we go through the final hours of the launch countdown. At this point, the Mission Management Team has really done their job and we've handed off responsibility for the successful launch of the mission to the launch director who is directing the Launch Control Team, as long as he or she is able to work within the constraints of the Launch Commit Criteria. That is huge, several-volume book which is the what-if of every launch and represents the corporate history of all the problems that we've either experienced or we've had the opportunity to think through ahead of time that we might experience and our reactive measures that we would take to further clarify the problem and our ability to go launch safely. For practically all the launch commit criteria, when you run through the procedures, it ends up in one or two branches. Either you have resolved the issue as being safe to go fly, clear to launch or, no, we're not sure, you have to stand down that day, unless the Mission Management Team is offered rationale which allows you to press forward and approves it in real time. The Mission Management Team is there to provide guidance if the launch director gets outside the launch commit criteria and needs guidance. GEN. DEAL: Jim, I just want to get back to in-flight anomalies very quickly and get your perspective because you experienced a very serious one on STS 83 personally. What I want to do is get your perspective on, following STS 83, how the process went, did it underscore the strengths in the program, or were there lessons learned by which we improved the in-flight anomaly process following STS 83. COL. HALSELL: Certainly I can lend my experience from STS 83, and I think the question that you're asking about the in-flight anomaly process is one of the reasons that we invited Bob Castle, as one of the representatives of the in-flight MMT team, to comment. So I'll hand off the remainder of that question to him. The issue you're talking about on STS 83 back in 1997 was that after we launched, we experienced an in-flight anomaly concerning some out-of-family and unacceptably divergent fuel cell substack delta volt readings, which is a way of saying there were some increased level of risk that if we were to continue the mission with that fuel cell powered up that you could experience crossover and that could lead to fire and/or explosion. So that was deemed to be an unacceptable risk. It was equally unacceptable to shut down and save that fuel cell and continue the mission to nominal conclusion on just the two remaining fuel cells. So the Mission Management Team came to the conclusion that the only safe and prudent thing to do was to have us close up the lab, prepare to make an early entry back home; and we did so after only four days in space. The conclusion of that story is that between then and STS 84 which, as I remember, wasn't the very next but the one-after-the-one-after flight, on STS 94, they resolved that particular issue, they understood it after they were able to get the fuel cell and do all the testing back at the vendor to understand that, in fact, it had most likely been an indication problem, not an actual issue, and that we could have stayed up on orbit. But there was no way to have known that in real time and I, certainly as the recipient of the safest course of action, I appreciate the action that the MMT took at that time. So I think that is an example of how, when faced with extremely difficult choices, expensive choices both in terms of money, in terms of the manifest having to be replanned for probably several years downstream, but still when confronted with that highly undesirable set of consequences for making the safe decision, the on-orbit Mission Management Team did make that decision. They brought us home and we re-flew that mission a couple of flights later with a full measure of success. ADM. GEHMAN: Okay. Let's let Mr. Castle give his introductory remarks, and we can always ask questions later. MR. CASTLE: Okay. Well, that does lead into what I was going to start talking about a little bit. I don't have any charts. So you can feel free to interrupt me even more freely than you have already. As far as the way the realtime team goes, we pick up the launch. Right after liftoff is when the realtime team picks up and starts conducting the flight. I would call flight director the mid-level management team that Jim referred to. The flight director also has his set of requirements. The specific ones that come to mind are the flight rules and the SODB, which is the Shuttle Operational Data Book. The flight rules is a large book. I didn't bring one around. It's about yea thick for the space shuttle. It's what I call pre-made decisions, decisions you've already done your what-if'ing and you've thought about them and you've thought about the situations and the cases very carefully and you write down what it is that you're going to do for each of these particular cases. In the one that Jim mentioned, the loss of one fuel cell, it says you need to land what's called a minimum duration flight to minimize the length of time we stay in orbit because if you lose another fuel cell, you can land with only one fuel cell but the power-down you have to get into is dramatic and it impacts your avionics in lots of other ways. So we've already gone through that debate. If we lose one fuel cell, we're going to land and we're going to cut the flight short, early. The MMT got involved with his flight because it wasn't really clear from the indications whether we really had a bad fuel cell or not. So that's where we had to call the engineering guys together to look at that. But if it's clear we've lost a fuel cell, the flight control team doesn't have to consult anyone. We'd say, okay, the flight rules say go do this, so this is what we're going to go do. The SODB is the Shuttle Operational Data Book. That is another book that is maintained by the space shuttle program. It's a list of how you operate the shuttle. You can operate the shuttle with the temperatures on this loop, greater than this and below that. This type of information. Kind of like an owner's manual for your car except, again, it's several volumes. It's fairly thick. The flight rules are controlled by the shuttle program. The final version of all of them are taken forward to the PRCBs for approval. There are several lower-level boards chartered by the program that manage those rules. People have asked about the safety process. Any changes to the rules, that's done on what's called a CR form, a change request. The Safety folks review those as well, as all the rest of the disciplines -- engineering, program offices, space and life sciences, FCOD, MOD, all the different areas. There is a mid-level board, what's called the Flight Rules Control Board, which is chaired right now by one the deputy chiefs of the Flight Director Office. Again, all of those same organizations represented and then their approved set of rules come forward in a change package to the PRCB for final approval by the program. A very similar process used for the SODB, the way it's managed. So those are two things that I start off with as my requirements, if you will. There are a couple of other things that are like the flight requirements document which are a mission-specific document. Okay. The other two I just mentioned, that's how you operate the orbiter, how you fly. The FRD says, well, here's what we want you to go do. We want you to conduct a space lab mission. Here's how long we want you to stay in orbit. Here are the priorities of things we'd like you to do. That type of information. There is also a much smaller book of flight rules that are flight specific. In that again, you're writing down rules, mainly a priority list, rules that are specific to the payload or the particular operation you have on that flight. Those are flight specific. Also approved by a very similar process and finally approved by the shuttle program manager at the PRCB. Also I want to say that the flight rules are things that when we train people, we take these things very, very seriously. The simulation folks try to put in failures and various scenarios that will stress people's thinking. Okay? They'll break a piece of instrumentation someplace in the simulator. Well, do people recognize what's just failed? Do they recognize the instrumentation they've lost? Do they understand the implications to the flight rules? Have you just had a flight rule violation because of this failure? Sometimes just loss of instrumentation is no big deal. Sometimes you really have a rule violation because we've thought through if I don't have this measurement, then this thing that's really bad can happen to me and there's nothing I can really do to detect it or I've actually impacted the safety of the vehicle because I can't measure something. Sometimes they don't. Each rule is also annotated. Let me back up. Jim talked about the top-down hazard process and the bottoms-up failure modes and effects process. Anytime that this hazard control process says we need to control this hazard by a certain operational constraint, we want you to always flip this switch before you flip that switch, a flight rule gets written that says always do it in this order. That flight rule gets annotated that it's a hazard control. So anybody reading the rule book knows that this is a control for a hazard that's been identified for the program. That does a couple of things. The main thing it does for you is when somebody comes along and says I'd like to change this rule for whatever reason, it's in black and white, right in front of you, that you've got to run this by the safety community, you've got to look at it carefully, look up that hazard control, make sure you're not undoing what we carefully did. They're also flagged from the bottoms-up review. Anybody in the bottom-up review that comes up with a classification of either a Crit 1, 1R, 1S, and 2, I believe, gets classified on a Critical Items List or a CIL. So we flag those rules, as well. It says, okay, this rule is part of the rationale for saying this critical item is acceptable. Again, you get the same type of things that are controlled operationally. If you have Failure A, then you must take this following action to make sure another problem doesn't sneak up on you. Everybody works really hard to understand those, even though the book is very, very thick. We train them very, very heavily. Our simulation guys are very sneaky. They will put in an instrumentation failure here and a power system failure there and an avionics box failure here and you've got to realize that when you add all those three things up, you've really got a much more serious problem than it seems like. Generally they'll set us up that you need to recognize, hey, one more failure could really be bad. So we work that very, very hard. Again, I'm just going to keep talking until somebody wants to stop and ask me questions. Let's see. The basic rule, again, is the flight rules and the SODB -- when I say the realtime team, let me talk a little bit more about who the realtime team is. There is the Flight Control Team, which is led by the flight director who sits in the middle of the room. I don't have a picture, but you've seen the room. There are flight directors there 24 hours a day during a shuttle mission. We also appoint a lead flight director who is appointed generally at least on the order of a year before the mission. They oversee not only the mission but all the launch preparation, all the preparation times, all the crew training, everything else that goes on for that prior year. That includes the little first chart that Jim put up, all the little boxes. Either the flight director or some member of his team plays in every one of those boxes throughout the preflight process. There are other members of what I call the flight control or the realtime team. A very important team is the MER, the Mission Evaluation Room. That is a room that's down on the first floor of Building 30. It is run by the program office and is staffed mainly by people out of engineering and various contractor support -- Boeing, various subsystem contractors. Their function is evaluation. They watch what's going on on the vehicle. They look for more shuttle trends, things aren't clear black and white but maybe more subtle problems. If there is a problem, of course, they're ready to be activated, ready to go work any details. Things are never quite as crisp and clean as they look like in simulation. So you always like to have the engineering talent there, ready to go. That group in the MER includes a safety console position, again, always watching what's going on as we operate the mission, understanding all the hazard controls and all the things that have been preflight analyzed. There's another room in the building which is called the Customer Support Room and that is a program office room. Representatives who report directly to the program manager staff that room hours a day. Again, they're watching out for programmatic requirements. They're there to be consulted. If we get into a situation where I can't do what their priority list says I need to do, they're there to go rework that. "Okay. This just happened. I can't do your No. 3 item on the priority list. What would you like to do? What options would you like to invoke?" So they're there 24 hours a day, 7 days a week during the mission, ready for consultation; and they pay attention pretty well. There's a formal CHIT system. It's called a CHIT. I don't know what CHIT stands for, but there's a formal paperwork system where if we make a request for information or a request for special analysis, we write down exactly what we want. It is coordinated through the appropriate person who we're requesting this of. Anyone in the building can write such a CHIT. It comes down with an answer, and we don't close that CHIT until the originator agrees that whatever they wanted done has been done and done correctly. Again, it's a very formal process, I think, that works fairly well. ADM. GEHMAN: Let me interrupt. To carry over the discussion, I asked a hypothetical question, as did Dr. Logsdon, that if Columbia had returned safely from this mission, we still would have an IFA of a major foam strike. MR