|University of Bielefeld - Faculty of technology|
Networks and distributed Systems
Research group of Prof. Peter B. Ladkin, Ph.D.
|Back to Abstracts of References and Incidents||Back to Root|
|This page was copied from: http://catless.ncl.ac.uk/Risks/15.73.html|
ACM Committee on Computers and Public Policy, Peter G. Neumann, moderator
While I was in Copenhagen earlier today, a Danish friend, who knows of my interest in the A320, drew to my attention an item in today's issue of the news magazine "Goddaj" (if I recall the spelling correctly - it means "Good Morning"). A translation of the article follows (courtesy of my Danish friend):- --------Translation of Article in "Goddaj", 1st April 1994 -------- Danish Firm Scores Notable "First" ---------------------------------- Thor Avionics, one of Denmark's most advanced high-tech firms, has secured a contract which makes it the first software house in the world to provide "third party" maintenance on a major safety-critical software system. In order to reduce the maintenance costs on its fleet of Airbus A320 aircraft (the first type of civil airliner in the world to have a computer-controlled "fly-by-wire" system), Air France has placed Thor under contract to provide all future maintenance on the software of this highly-automated aircraft. Wolf Larssen, director of Thor, said "This is the first contract of its type, and it won't be the last. Users of commercial software long ago discovered that there are great savings to be made by getting a "third party" firm to maintain their software. I am only surprised that it has taken users of safety-critical systems so long to discover the advantages. I expect other A320 operators to be placing similar contracts before too long." A "third-party" in this context means a firm which is independent of both the user and the supplier. Such firms, being "lean and mean" are usually capable of providing a much better and more cost-effective service than the original supplier, since they have fewer overheads and are less stifled by bureaucracy. In the commercial world, such contracts have usually gone to small, dynamic, organisations, and it seems that the world of safety-critical software will follow suite. "We had to beat some stiff opposition from Sextant Avionique, Matra, Logica, and similar large firms." said Mr. Larssen. "The fact that the software on the A320 will need to be maintained indefinitely means guaranteed jobs for highly qualified Danish workers for a long time to come." M. Theophile Gautier, spokesman for Air France, said "We have the utmost confidence in Thor to deliver the goods, both in terms of reduced cost, improved system performance, and increased safety." The automated systems on the A320, particularly the flight control and flight management systems, have sometimes been called into question following the various accidents involving this type of aircraft, although the accidents have generally been ascribed to pilot error. Even so, there is an obvious question mark over the ability of a third-party firm to maintain the level of safety. When asked about this, Mr. Larssen said "Our software maintenance and validation process is second to none. Although Airbus Industrie have refused to release the source code, so that we will have to strip out the binary and work from that, we anticipate no problems. Most of the modifications we will be making are fairly slight, so that regression testing can easily be done on a software flight simulator running on an Apple MacKintosh." A spokesman for the JAA (Joint Aviation Authority, which is responsible for certifying that any new or modified design of aircraft is airworthy) said "The basic design has already been certified. All that Thor will be doing are minor post-certification modifications. Thor themselves have been certified as conforming to the ISO-9000 quality standard and to SEI level 2, so it should not be difficult for them to meet the requirements for our own certification, which is based upon an industry standard referred to as RTCA-DO/178B." In response to questions about what the maintenance would actually involve, Mr. Larssen said "Occasionally, Airworthiness Directives are issued by the JAA which require changes to be made to the design of an aircraft in order to correct a fault. Where this change involved modifying the software, Thor will be responsible for doing this. The beauty of software is that the modified version can be installed on all existing aircraft in seconds, simply by inserting a new eprom. In addition to this corrective maintenance, we will also be offering Air France enhancements to improve the performance of the A320. The practice of "chipping", or modifying the firmware in the engine management system of an automobile such as a BMW in order to make it go faster, is well established. I don't expect that we could make your A320 perform like an F-111, but we could certainly extend the "safe flight envelope" beyond the rather conservative limits originally set by the manufacturer." -------------------- Article Ends ------------------------ I leave it to readers to draw their own conclusions! Peter Mellor, Centre for Software Reliability, City University, Northampton Sq London EC1V 0HB +44 (71) 477-8422, email@example.com [This is quite a Thor-ny piece. Incidentally, I note that "goddaj" is really "good day" (albeit used in the morning, as in the case of Guten Tag), and April 1 is certainly a "goddaj". Unfortunately, occasional adjacent-key typing errors might easily replace the "j" with an "m", which might be an appropriate reaction. PGN]
> NAUSEA for NASA. Singularly appropriate some days. Microsoft Word's persistence in attempting to substitute Colada for Collider certainly made me feel the need for a drink when writing about the SSC... --JOe
The main risk is in relying too heavily on spell-checkers. As people produce more of their own documents, they no longer have someone who does most of the proof-reading, and rely on a program instead. Automation of other parts of document production has caused a change in the type of errors that can get through. Up until a few years ago, most errors in trade books were switched letters ("b" for "d") probably caused by manual typesetting. Now one finds many more mistakes of a wrong word, no doubt from a spell-checker substitution. Perhaps we can ask, who checks the spell-checkers? -michael j zehr
By definition, a spell checker is a product which eliminates a large set of errors in a text. It does not eliminate them all. I would suggest that you do not go onto "auto pilot" when using the spell checker. Instead use the same level of awareness that you do when your write. In fact it makes sense to examine the text around every place the spell checker stops. There are a lot of errors which can only be eliminated by human attention. As far as I can see your general problem would not be eliminated by getting rid of profanity. Suppose you had a "Ms. Gorse" in your document. A spell checker might offer "Goose". Your client (or boss) might be equally offended.
A company which sometimes competes with my employer sells a software package which includes a spelling checker. It flags <our product> as a misspelling and offers <their product> as the suggested alternative. The RISKs? None that I can think of, but it's a nice anecdote. Eric Sosman Interleaf, Inc. / Prospect Place firstname.lastname@example.org 9 Hillside Ave. / Waltham, MA 02154 (USA)
Note this took out a reported 200+ DS3 circuits. That's ~~100,000+ voice-grade circuits (if all were such). Netcom's DC POP was one of the DS1's. They had leased the circuit from WilTel, but WilTel in turn had subcontracted the facilities from MCI. Further, while MCI had the cable back up by 11pm, somehow WilTel did not communicate this to Netcom. Thus the POP was not restored until the next morning. (Irony here - WilTel got started pulling fiber through abandoned oil pipelines. Schedule 300 pipe provides much better than average protection against backhoe fade.) Classic RISKs: 1) Too many eggs in one basket. While MCI surely has reserve capacity, it does not seem to have 200 DS3's worth. No self-healing ring, it seems. 2) Lost-in-translation syndrome - Once more than two organizations are involved, the chances of getting any intact message from one end to the other goes down as an exponential function of the number of hops. ps: Ispell wants to turn "WilTel" into "Wilted"........
It is interesting that both of these incidents have a common thread - no feedback loops. Way back in the '70s when I was part of the team that designed the full authority digital flight control system for the AFTI F-16, we had a similar problem: the system was so complex and so many people were involved that it was easy to miss the change that Jon made today would affect Harold's system - and this was during the design stage. In production, component substitution could have the same effect, some so subtle that it would not be noticed until a pilot found himself in an interesting situation. One of my tasks was to develop the simulation software used in a 40 foot Evans & Sutherland dome & as such with each revision of the flight control software, the appropriate changes had to be fed into the dome system. In order to maintain continuity we developed a "configuration control model" that simply scanned the source code for all uses of a variable or subroutine and provided a map of the points of contact for each variable. When a change occurred, it was a simple matter to report the change to each affected engineer/programmer. It was also an excellent tool for reporting when someone had accidentally used the wrong variable in an equation since it would suddenly show use in a routine it had not been used in before. This tool also made it possible to notify those responsible for affected modules when a component change was made since the tree for the variables used with the component was readily available. The process was really simple but deductive rather than inductive: changes were detected not by people submitting a change notice but by a comparison of "current" versus "last", active configuration management rather than passive. Several times changes were found before the paperwork arrived. The simple fact is that any large system, from a telephone number list to aircraft fight controls is subject to Chaos math: small omissions over time will increase in effect. Murphy says that unknown effects will be destructive. Multiple omissions multiply effects. The most effective answer I have found is active feedback loops, something computers are very good at. Today one way I protect sites from intruder attacks is by requiring modem registration and briefing of owners. I also conduct random sweeps of the telephone lines looking for unregistered modems. Without the second, the first would rapidly become obsolete. This has two advantages: 1) I find omissions quickly. 2) People are less likely to make omissions knowing that they will be noticed. Over the last few years I have seem many instances in RISKS of problems with aircraft flight controls making the wrong decision or telling the pilot the wrong thing and each time have wondered if active design or configuration management feedback loops could have prevented them. Padgett
I saw on San Francisco's channel 4 last night that a jail in Marin which houses such people as Polly Klaus' killer has been having problems with their cell doors. Apparently, without reason, they would just open. The prison said their was no danger in escape. They blamed the problem on "software errors". How about that? --tom
The RISKS archives include the following items from the ACM SIGSOFT Software Engineering Notes (S vol i no j). Recent items also appear in the on-line RISKS. PGN ..... Prison problems Seven Santa Fe inmates escaped; prison control computer blamed (S 12 4) Oregon prisoner escaped; frequent-false-alarm alarm ignored (S 12 4) New Dutch computer system frees criminals, arrests innocent; old system eliminated, and no backup possible! (S 12 4) New El Dorado jail cell doors won't lock -- computer controlled (S 13 4) San Joaquin CA jail doors unlocked by spurious signal; earlier, inmates cracked Pelican Bay State Prison pneumatic door system (S 18 2:4)
Man, just when I thought I understood this stuff. I have condensed this down to the following: euclid% euclid% mkdir something_scwewy euclid% cd !$ euclid% foreach i (a b c d) ? echo $i > $i ? end euclid% find . -type f -print | xargs -n1 more ./b ./c ./d --More--(Next file: ./a) # Hit <SPACE> ./a :::::::::::::: a euclid% Now, to my way of thinking, it should be executing the commands "more ./a; more ./b; more ./c; more ./d". Certainly I have had and come to expect this sort of behavior from xargs in the past. It seems to be a problem with "more", because I get decent behavior with, say, "echo" and "cat": euclid% find . -type f -print | xargs -n1 cat a b c d Yet: euclid% find . -type f -print | xargs -t -n1 more more ./a ./b ./c ./d BTW, if there are more than a screenful of files, I get prompted by more to scroll through the list of them before it actually runs more on the first file. I don't get this at all. This is on SunOS 4.1.3. Peter Scott, NASA/JPL/Caltech (email@example.com)
This is an example of a strange interaction of two bugs, one in `more' and one in `xargs'. All bugs are RISKS to some extent, its not clear how severe or unusual they need to be to make it into RISKS... There are two strange things occurring here. 1. When `more' is invoked with its standard input connected to something OTHER than a terminal, it treats `stdin' as the first file to display. 2. `xargs' doesn't close the input to the child it invokes. So what happens is, `xargs' invokes `more ./a', and `more' reads everything it can from its standard input, which connects to the `find'. When `more' finishes, `xargs' finds that its `stdin' is empty and exits. To exercise these bugs separately, try: echo a b c | more ./a echo a b c d | xargs -n1 cat - Chris Dodd firstname.lastname@example.org
connection to the Internet (CHINANET; sub CHINANET to LISTSERV@TAMVM1.TAMU.EDU). The Chinese have named their new project to connect China to the Internet the "Golden Bridge" project. The following document purports to be the newly developed "PRC Regulations on Safeguarding Computer Information Systems." It seems quite appropriate for RISKS. As you read this, keep in mind that 1) in China accused persons are guilty until proven innocent; 2) laws referred to in the document as ones applying in certain circumstances are often harsh, subject to change without notice, and so vaguely worded as to make easy the prosecutor's job, not of proving guilt (not necessary), but of arguing why the penalty should be maximized; 3) the "Public Security" laws referred to are the same laws that stipulate that the families of serious offenders will be billed for the single bullet used in judgement; 4) certain concepts (virus, special security products) are either poorly defined or all inclusive; 5) in China when there is doubt as to the legality of any particular act, illegality is assumed (this is important not only in court, but also in normal life, where people tend to be more conservative in part because of it.) As we welcome this brave new domain into our net.universe, it will be interesting, and perhaps surprising at times, to see how another set of explorers on the electronic frontier are approaching the flow of information. Golden Bridge, indeed. As read, sending email without filing a customs declaration, or accepting a shareware registration for an anti- virus product could both be construed as being illegal. There's a lot of room for improvement here, imho. =============================================================== P.R.C. Regulations on Safeguarding Computer Information Systems =============================================================== Source: Beijing XINHUA Domestic Service in Chinese, February 23, 1994 From: email@example.com (John Ho), Asia Online Chapter I. General Provisions Article 1. These regulations have been formulated to safeguard computer information systems, to promote the application and development of computers, and to ensure smooth progress in socialist modernization. Article 2. The computer information systems referred to in these regulations are man-machine systems, composed of computers and their allied and peripheral equipment and facilities (including networks), that collect, process, store, transmit, and retrieve information according to prescribed goals and rules of application. Article 3. In safeguarding computer information systems, measures shall be taken to secure computers, allied and peripheral equipment and facilities (including networks), the operating environment, and data, as well as to ensure the normal functioning of computers, so as to safeguard the safe operation of computer information systems . Article 4. In safeguarding computer information systems, priority shall be given to the security of computer systems containing data on such important areas as state affairs, economic construction, national defense, and state-of-the-art science and technology. Article 5. These regulations shall apply to safeguarding computer information systems within the PRC's borders. Measures for safeguarding microcomputers that have not been hooked up shall be enacted separately. Article 6. The Ministry of Public Security shall be in charge of safeguarding computer information systems. The Ministry of State Security, the State Secrecy Bureau, and relevant State Council departments shall carry out work pertaining to safeguarding computer information systems within the lines of authority prescribed by the State Council. Article 7. No organization or individual may use computer information systems to engage in activities that endanger national or collective interests, as well as the legitimate interests of citizens; they may not jeopardize computer information systems. Chapter II. The Safeguards System Article 8. Computer information systems shall be established and applied in accordance with laws, administrative rules, and relevant state provisions. Article 9. Computer information systems shall be protected on the basis of security grades. The Ministry of Public Security, in conjunction with relevant departments, shall establish security grades and formulate specific measures for protection based on such grades. Article 10. Computer rooms shall conform to state norms and relevant state provisions. No work may be carried out in the vicinity of computer rooms that jeopardizes computer information systems. Article 11. Units using internationally networked computer information systems shall register their systems with the public security departments of people's governments at or above the provincial level. Article 12. Individuals who ship, bring, or mail computer information media into or out of the country shall file truthful declarations with the customs authorities. Article 13. Units that use computer information systems shall establish security management systems and assume responsibility for safeguarding their computer information systems. Article 14. Units that use computer information systems shall report any incidents relating to their systems to the public security departments of local people's governments at or above the county level within 24 hours of the incidents. Article 15. The Ministry of Public Security shall exercise centralized management over research into the control and prevention of computer viruses and other harmful data that jeopardizes public security. Article 16, The state shall implement a licensing system for the sale of special safety products for computer information systems. The Ministry of Public Security shall enact specific measures in conjunction with relevant departments. Chapter III. Supervision Over Security Article 17. Public security organs shall perform the following functions to supervise efforts to safeguard computer information systems: (1) Supervising, inspecting, and guiding the work of safeguarding computer information systems; (2) Investigating and dealing with illegal and criminal cases involving the endangerment of computer information systems; and (3) Other supervisory functions with regard to safeguarding computer information systems. Article 18. Upon detecting latent hazards in computer information systems, public security organs shall promptly advise the units that use such systems to institute safety measures. Article 19. Under urgent circumstances, the Ministry of Public Security may issue special circulars on specific security aspects of computer information systems. Chapter IV. Legal Responsibilities Article 20. In the event of any of the following violations of the provisions in these regulations, public security organs shall issue warnings or shut down the computers for screening purposes: (1) Contravening the system for protecting computer information systems based on security grades and jeopardizing computer information systems; (2) Violating the registration system for internationally networked computer information systems; (3) Failing to report incidents related to computer information systems within the prescribed time frames; (4) Failing to take remedial action within the prescribed time after receiving notification from public security organs mandating security improvement measures; (5) Other actions endangering computer information systems. Article 21. Public security organs, in conjunction with relevant units, shall deal with cases in which computer rooms do not conform to state norms or relevant state provisions, or in which work carried out in the vicinity of computer rooms endangers computer information systems. Article 22. The customs authorities shall deal with failure to file truthful declarations on computer information media shipped, brought, or mailed into or out of the country, pursuant to the "PRC Customs Law" and the provisions outlined in these regulations and other laws and regulations. Article 23. Public security organs shall issue warnings or impose fines of not more than 5,000 yuan and 15,000 yuan, respectively, on individuals or units if computer viruses or other data harmful to computer information systems are deliberately input into such systems, or if special safety products for computer information systems are sold without permission. They shall confiscate illegal proceeds and impose a fine that is 100 or 300 percent more than the sum of such proceeds. Article 24. Actions that violate the provisions in these regulations and constitute infractions of public security shall be punished pursuant to relevant provisions in the "PRC Regulations on Security Administration and Punishment"; if the actions constitute a crime, criminal responsibilities shall be investigated. Article 25. Any organization or individual who inflicts property losses on the state, collectives, or other individuals in violation of the provisions in these regulations shall assume civil responsibility in accordance with the law. Article 26. Interested parties who are dissatisfied with specific administrative actions carried out by public security organs pursuant to these regulations may apply for administrative reconsideration in accordance with the law or file administrative lawsuits. Article 27. Government functionaries who abuse their power to demand and take bribes or commit other illegal or delinquent acts while enforcing these regulations shall be punishable on criminal grounds if their actions constitute crimes or given disciplinary actions if their actions do not constitute crimes. Chapter V. Supplementary Provisions Article 28. The meanings of terms used in these regulations are defined as follows: Computer viruses mean a set of self-replicating computer commands or programming codes inserted during the course of programming or into computer programs that can impair computer functions, destroy data, or affect computer use. Special safety products for computer information systems mean special hardware and software products for use in safeguarding computer information systems. Article 29. Military-related computer information systems shall be safeguarded in accordance with relevant military laws and regulations. Article 30. The Ministry of Public Security may formulate implementation measures in accordance with these regulations. Article 31. These regulations shall take effect upon promulgation.
|This page was copied from:||http://catless.ncl.ac.uk/Risks/15.73.html|
by Michael Blume