To: MAB Distribution
From: F. W. Martinson
G. C. Dixon
Date: June 11, 1980
Subject: TR Closure and Resolution
PLEASE REMOVE THIS COVER LETTER BEFORE FILING THIS MAB.
This MAB describes changes to the current method for closing
trouble reports. The new method of closing TRs brings Multics
definitions of TR response and resolution closer to those used in
goaling TR performance with other Honeywell operating systems.
It also addresses several outstanding problems in our current
method of resolving reports. This includes the following
problems:
(a) developers cannot give accurate release numbers (in which
their software fix will actually be installed) when they
answer a TR, due to the uncertainties of the release naming
and generation process.
(b) TR responses are not currently linked to changes in bug lists
associated with the various areas of the system.
(c) TRs which cannot or will not be fixed are not properly
handled by the current TR mechanism. The aspect of
documenting such system limitations is not currently
addressed.
(d) there are too many TR states, and the transitions between
states is confusing. This makes it difficult for developers
to know how to proceed.
Please take particular note of the following changes.
(1) The definitions of a TR response and a TR resolution have
been changed.
(2) A new state code called "limitation" has been created to
replace the old no_fix code. Along with "limitation" is a
new procedure for resolving system limitations which ensures
that the limitation is documented.
(3) Renaming of the bug and not_bug state codes to "error" and
"not_error". The term bug proved to be a misnomer for
documentation errors.
(4) Elimination of the not_repeatable state code. Most people
(especially submitters) thought that non-repeatable problems
really required more information from the submitter, and
should therefore be classified with the "needs_info" state
code.
(5) Elimination of release numbers in TR answers. No commitment
will be made in TRs to fix a problem in a given release.
(6) Collapsing of the large number of submitted_XXX and
installed_XXX state codes into a single "submitted" code.
(7) Creation of error lists covering all areas of the system.
(8) A new requirement that error list name and number be given
with all answers using the error, change_pending, limitation
or submitted state codes.
(9) A new facility to answer TRs automatically, based upon
changes made by developers to their error lists.
This MAB proposes that an interface be created between errors
lists and the TR system. This new facility would make it easier
to respond to TRs by generating automatic TR responses and
resolutions, based upon changes which the developer makes to an
error list. This MAB represents a commitment to provide such an
interface, but does not propose any specific interface. The
actual interface to be used is the subject of an upcoming MTB.
The changes in this MAB will go into effect on or about July 1,
1980.
MULTICS ADMINISTRATIVE BULLETIN MAB-049, Revision 1
To: MAB Distribution
From: F. W. Martinson
G. C. Dixon
Date: June 11, 1980
Subject: TR Closure and Resolution
INTRODUCTION
This bulletin formally defines the criteria for verifying, responding
to and resolving trouble reports (TRs). It also defines standard
answers to be provided, based upon the new state given in an answer to
the report. The processing of TRs is explained in more detail by the
scenarios given in Appendix A.
TROUBLE REPORT PROCESSING STAGES
Three kinds of trouble reports can be submitted by a user or SiteSA:
problem reports, suggestions and questions. This MAB is concerned
primarily with problem reports. Problem reports are processed in
three stages: verification, response and resolution.
The verification stage involves investigation of the problem report by
the TR Administrator (who is responsible for initial processing and
routing of TRs) or by another member of the Multics Software Support
unit. Investigation ensures that the problem can be reproduced, and
that it is not a system limitation.
In the response stage, the developer acknowledges that a problem
exists, and adds the problem to an appropriate error list (formerly
called a bug file). Similarly, the developer can (by updating the
appropriate error list) define the problem to be a system limitation
which will be documented in future manuals. The TR Administrator or
investigator can respond to a problem or limitation that already
appears in an error list.
In the resolution stage, the TR Administrator or investigator
indicates that a problem report is not understood, not repeatable or
not really a system problem. In such cases, the resolution asks for
additional information about the problem, or explains why the reported
item is not a problem.
In another kind of resolution, the developer indicates (by updating an
error list) that a problem fix has been submitted for installation in
the system libraries (not including the EXL library); or the technical
writer indicates (by updating an error list) that the online (ie,
compin segment) version of the affected manual contains necessary
corrections to fix a documentation problem, or to document a system
limitation.
The current stage of processing of a report is indicated by its stage
code. There are four codes: ENTERED, VERIFIED, RESPONDED, and
RESOLVED. These define the boundaries between processing stages, as
shown in the figure below.
Stage Code Processing Stage
---------- ----------------
ENTERED
\
> verification
/
VERIFIED
\
> response
/
RESPONDED
\
> resolution
/
RESOLVED
TR RESPONSE AND RESOLUTION STATE CODES
There are twelve states that can be given (via the
answer_trouble_report command) to respond to or resolve a trouble
report. These state values are mapped into a standard answer text by
the tr processing commands when the answer is sent to the submitter.
Thus, the state values are a shorthand method of providing a standard
answer text.(1)
______________________________________________________________________
(1) Note that the answer_trouble_report command can still be used to
provide information in addition to these canned answers. The
canned answers represent the minimum answer text which will be
provided for TRs.
______________________________________________________________________
The state values also change the stage of processing for a report,
moving a report from one stage of processing to the next. The
relationship between state values and stage codes, and the standard
answer text for each state value are shown in the table below. Refer
to the TR processing scenarios in Appendix A for a more thorough
description of the TR process, and the particular actions that change
a TR from one processing stage to another.
State Value Processing Standard Answer
Stage
-------------- ----------- ---------------------------------------
investigating ENTERED Thank you for your report. It is
currently under investigation.
verified VERIFIED Your report has been verified and
forwarded to the developer for
consideration.
error RESPONDED This has been noted as problem number
NNN on the EEE error list. See error
list entry.
change_pending RESPONDED This has been noted as problem number
NNN on the EEE error list. A fix is
known and will be included in a future
release. See error list entry.
limitation RESPONDED This has been noted as problem number
NNN on the EEE error list. The problem
will be documented as a system
limitation. See error list entry.
submitted RESOLVED The fix for problem number NNN on the
EEE error will be available in the next
possible release or documentation
update. See error list entry.(1)
needs_info RESOLVED We cannot reproduce your problem.
Please supply more information.
not_error RESOLVED Your report does not describe a
problem.
answered RESOLVED The answer to your question follows:
entered RESOLVED Your suggestion has been forwarded for
consideration.
accepted RESOLVED This suggestion has been accepted and
will be implemented as time permits.
rejected RESOLVED This suggestion will not be
implemented.
______________________________________________________________________
(1) The error list entry for a response having the submitted state
could indicate that the problem had actually been fixed in a prior
Multics release. It could also provide bypass techniques, etc,
just as error lists do today.
______________________________________________________________________
TR PROCESSING PERFORMANCE MEASUREMENTS
The goals for TR processing performance are discussed in a separate
MAB (MAB-044). Measurements will be made periodically to determine
how well TR processing meets these goals. Only external trouble
reports are considered when measuring for goaling purposes. External
reports are those entered with a site field other than System_M or
MIT. The time periods used in goaling measurements are defined as
follows:
Time between ENTERED and VERIFIED = Verification & Routing
Time between ENTERED and RESPONDED = Response
Time between ENTERED and RESOLVED = Resolution
EXISTING TR BACKLOG
All TRs entered under this redefinition of states will be assigned
numbers starting with 7000. Only TRs from 7000 on will be used for
measurement purposes and statistical reporting using the criteria
defined above.
Old TRs will be processed, using the new algorithms and procedures,
but will be considered resolved in terms of the TR performance goals.
However, problems in older TRs will still be tracked and fixed.
Old TRs will be converted to a new format and stored in the new TR
database (currently under development) as they are processed by the TR
Administrator. Prior to being entered in the new database, they will
be stored in the existing TR archives.
MANAGEMENT REPORTS
Use of the new state codes and the new TR database facilitates the
generation of reports management has been requesting to track
TRs/errors against goals. Periodic reports describing TR processing
performance will be generated for each management unit.
ERROR LISTS
A key requirement of this plan is the formulation of an interface
between the TR system and the error lists maintained by developers.
This interface is the subject of an upcoming MTB.
Error lists (formerly called bug files) will be required for all areas
of the system. There will probably be a separate error list for each
Priced Software Product (PSP), plus a small number of additional lists
for non-PSP software.
Many developers now maintain error lists in a variety of formats using
a variety of tools. Provision of a standard interface between error
lists and the TR system allows automatic generation of TR responses
and resolutions whenever a developer updates an error list.
Of course, not all responses or resolutions can be generated
automatically. Some will require supplying additional information to
describe the particular error, or an error bypass, etc. Other
resolutions may not involve creation of an error list entry (eg,
needs_info, not_error, answered). The answer_trouble_report command
can still be used to enter such responses and resolutions.
APPENDIX A
TR Processing Scenarios
The scenarios which follow illustrate typical patterns which can occur
in processing of Trouble Reports. They are not intended to indicate
required steps in processing. Some of the steps in the illustrations
could, for example, be skipped when processing a particular TR.
Rather they are an attempt to describe the kinds of action taken and
decisions made to propel a TR through the TR processing mill.
Please note the following:
(1) The scenarios below do NOT show all possible state transitions.
They show some typical state transactions, but many other possible
transitions exist.
(2) A (loop-back) change from one of the RESOLVED state codes (such as
needs_info) back to an ENTERED, VERIFIED or RESPONDED state code
(such as entered or verified) is possible. Consider the TR
PROCESSING FOR AN UNCLEAR REPORT, on page A-7.
_________ TR PROCESSING FOR A DOCUMENTATION PROBLEM
| |
| START |
|_______|
|
| Submitter uses epr to enter problem report.
____V______
| |
| ENTERED | (entered)
|_________|
|
| TR Administrator verifies that documentation
| problem exists, and is not already a known error
| (as defined in a documentation error list). TR
| is sent to technical writer.
____V_______
| |
| VERIFIED | (verified)
|__________|
|
| Technical writer acknowledges error by adding an
| entry referencing this TR to error list for the
| affected manual. This change to error list
| automatically generates a response to the TR.
____V________
| |
| RESPONDED | (error)
|___________|
|
| Technical writer marks error list entry,
| indicating that the online version of the
| affected manual has been corrected to fix the
| error. A TR resolution is automatically
| generated from the change to the error list.
____V_______
| |
| RESOLVED | (installed)
|__________|
-----------------------------------------------------------------
|
LEGEND: | Action Taken
____V__________________
| |
| TR PROCESSING STAGE | (state value)
|_____________________|
-----------------------------------------------------------------
_________ TR PROCESSING OF A SOFTWARE PROBLEM
| | (ERROR TO BE FIXED)
| START |
|_______|
|
| Submitter uses epr to enter problem report.
____V______
| |
| ENTERED | (entered)
|_________|
|
| TR Administrator investigates report or sends
| report to MSS personnel for investigation.
____V______
| |
| ENTERED | (investigating)
|_________|
|
| Problem is verified as reproducible and not a
| known system limitation (as defined in an error
| list). TR Administrator sends TR to developer.
____V_______
| |
| VERIFIED | (verified)
|__________|
|
| Developer acknowledges error by adding an entry
| referencing this TR to the appropriate error
| list. This change to error list automatically
| generates a response to the TR.
____V________
| |
| RESPONDED | (error)
|___________|
|
| Developer marks error list entry, indicating that
| a fix for the error is known, and committing to
| fix the error in a future release. A TR response
| is automatically generated from change to error
| list entry.
____V________
| |
| RESPONDED | (change_pending)
|___________|
|
| Developer marks error list entry, indicating that
| a fix for the error has been submitted for
| installation in the system libraries (not just
| in EXL). A TR resolution is generated from the
| change to the error list.
____V_______
| |
| RESOLVED | (installed)
|__________|
_________ TR PROCESSING OF A SOFTWARE PROBLEM
| | (ERROR IS A SYSTEM LIMITATION)
| START |
|_______|
|
| Submitter uses epr to enter problem report.
____V______
| |
| ENTERED | (entered)
|_________|
|
| TR Administrator investigates report or sends
| report to MSS personnel for investigation.
____V______
| |
| ENTERED | (investigation)
|_________|
|
| Problem is verified as reproducible and is not a
| known system limitation (as defined in an error
| list). TR Administrator sends TR to developer.
____V_______
| |
| VERIFIED | (verified)
|__________|
|
| Developer acknowledges error by adding an entry
| referencing this TR to the appropriate error
| list. This change to error list automatically
| generates a response to the TR.
____V________
| |
| RESPONDED | (error)
|___________|
|
| Developer decides error will not or cannot be
| fixed. He marks the error list entry,
| indicating that the error is a system
| limitation. A TR response is generated from the
| change to the error list entry. The TR is
| automatically sent to technical writer to
| document the limitation.
____V________
| |
| RESPONDED | (limitation)
|___________|
|
| Technical writer adds an entry referencing this
| TR to error list for the affected manual. In
| essence, it becomes an error that the limitation
| is not documented. Now the error appears in two
| error lists: in a software error list marked as
| a limitation; and in a documentation error list
| marked as a documentation error. The change to
| the documentation error list generates another
| TR response referencing the documentation error
| list entry.
____V________
| |
| RESPONDED | (error)
|___________|
|
| Technical writer marks the error list entry,
| indicating that the online version of the
| affected manual has been corrected to document
| the limitation. A TR resolution is generated
| from the change to the error list.
____V_______
| |
| RESOLVED | (installed)
|__________|
_________ TR PROCESSING OF A SOFTWARE PROBLEM
| | (ERROR APPEARS IN AN ERROR LIST)
| START |
|_______|
|
| Submitter uses epr to enter problem report.
____V______
| |
| ENTERED | (entered)
|_________|
|
| TR Administrator investigates report or sends
| report to MSS personnel for investigation.
____V______
| |
| ENTERED | (investigating)
|_________|
|
| TR Administrator or investigator notes that
| problem already appears in an error list, and
| responds to TR, giving error list name, error
| number and number of related TR (from error
| list). The related TRs become linked together
| in the TR data base so that an answer to one TR
| by the developer answers them all. The new TR
| then acquires the stage of processing and state
| value from the TR referenced by the error list.
____V________
| |
| RESPONDED | (error)
|___________|
|
| Developer marks error list entry, indicating
| that a fix for the error has been submitted for
| installation in the system libraries. A TR
| resolution is generated for the TR referenced in
| the error list, and all TRs linked to the
| referenced TR in the TR data base.
____V_______
| |
| RESOLVED | (installed)
|__________|
_________ TR PROCESSING FOR AN UNCLEAR REPORT
| |
| START |
|_______|
|
| Submitter uses epr to enter problem report.
____V______
| |
| ENTERED | (entered)
|_________|
|
| TR Administrator investigates report or sends
| report to MSS personnel for investigation.
____V______
| |
| ENTERED | (investigating)
|_________|
|
| Investigator cannot reproduce the problem, or
| does not understand the report. He uses atr to
| indicate that the report needs information.
____V_______
| |
| RESOLVED | (needs_info)
|__________|
|
| Submitter uses attr to provide additional
| information.
____V______
| |
| ENTERED | (entered)
|_________|
|
| Report processing cycle (and measured processing
| times) start over again with this new
| information. TR Administrator investigates
| report or sends report to MSS personnel for
| investigation.
____V______
| |
| ENTERED | (investigating)
|_________|
|
| Problem is verified as reproducible and not a
| known system limitation. TR Administrator sends
| TR to developer.
____V_______
| |
| VERIFIED | (verified)
|__________|
|
| Developer cannot reproduce problem, or cannot
| understand the report. He uses atr to resolve
| the TR by asking for more information.
____V_______
| |
| RESOLVED | (needs_info)
|__________|
|
| Submitter uses attr to supply further
| information.
____V______
| |
| ENTERED | (entered)
|_________|
|
| Again, the TR processing cycle (and measurement
| times) start over again. TR Administrator
| forwards report to developer (since it has
| previously been verified).
____V_______
| |
| VERIFIED | (verified)
|__________|
|
| Developer finds the problem described in report
| is caused by a user error. He uses atr to
| resolve the TR.
____V_______
| |
| RESOLVED | (not_error)
|__________|
_________ TR PROCESSING FOR A QUESTION
| |
| START |
|_______|
|
| Submitter uses eq to enter a question report.
____V______
| |
| ENTERED | (entered)
|_________|
|
| TR Administrator does not know answer to
| question, so she forwards TR to the developer.
____V______
| |
| ENTERED | (investigating)
|_________|
|
| Developer uses atr to resolve the TR by
| answering the question.
____V_______
| |
| RESOLVED | (answered)
|__________|
_________ TR PROCESSING FOR A SUGGESTION
| |
| START |
|_______|
|
| Submitter uses es to enter a suggestion report.
____V______
| |
| ENTERED | (entered)
|_________|
|
| TR Administrator sends suggestion to the
| appropriate developer. This resolves the TR.
| No further action is required. No commitment is
| made to accept or implement the suggestion.
____V_______
| |
| RESOLVED | (entered)
|__________|
_________ |
| | Developer accepts the suggestion by adding an
| | entry referencing this TR to the appropriate
| error list. The entry is flagged as a
O | suggestion to distinguish it from errors.
P | This change automatically generates a TR
T | resolution accepting the suggestion.
I | However, it does not constitute a commitment to
O | implement the suggestion. This step is
N | optional.
A ____V_______
L | |
| RESOLVED | (accepted)
S |__________|
T |
E | Developer marks error list entry, indicating
P | that the suggested change has been submitted for
S | installation in the system libraries on System
| M. A TR resolution reflecting installation of
| | the suggestion is automatically generated. This
| | step is optional.
| ____V_______
| | |
| | RESOLVED | (installed)
| |__________|
__|______