when do you know for sure

huangyhg

when do you know for sure?
this "class" of question has bothered me over the years:
we have an intermittent fault in some kind of a plant or equipment. failures occur sometimes once or twice a day and sometimes there are one or two months between failures.
there is no possibility to provoke the system, so an accelerated test cannot be done.
is there any standard method to have a "probable availability figure" and for how long does one have to observe the system to be able to produce such a number with, say, a 95 or even 99 percent confidence?
my thinking is that one would have to observe the system for many years to be able to say anything with some certainty. anyone has an idea about this?
check out our whitepaper library.
if you have historical data you can try to fit them to an exponential or weibull distribution.  then keep updating your curve fit as you get new data.  you will likely never know for sure the exact parameters of the distribution, since several values close together will likely yield the same approximate answer.  in that case, you can look for best and worst case responses to try and put boundaries on the parameters.
also, keep in mind that you may not be able to provoke the failure because you don't really know what is causing it.  there may be some as-yet unnoticed input or combinations of inputs providing the stimulus for failure, so don't count that out yet.
hope this helps!
thanks, bml.
that would help if i wanted to describe the distribution. and perhaps that is what i need to do to be able to find how many observations (or how long time) i need before knowing with some certainty that the problem has been eliminated.
i think that the distribution is purely random in most cases and it is for such a failure that i would like to know if there is any known method to tell the "boss" (customer, production manager, utility ceo etc) for how long one has to wait before one can say that "that's it". the only input in such a case is often notes saying that failures have occurred at that and that time.
we often do not have more than ten or twenty observations taken over one or two years and we want to tell the "boss" that we have to observe the system for x months before we can say that the measures taken to rectify the problem have had the desired effect. it is the "x" and the corresponding confidence number i am looking for.
you are probably right. we should plot the historical data to get some picture of the failures. problem is that i have no idea how to do that. our statistics course was rather meager, i'm afraid.
i agree with bml, a weibull distribution is probably the best (easy) way to examine this problem.
perhaps less daunting is just to work out the time between failure for each occurrence. if you then plot that as a distribution graph you will be able to see what shape the distribution is. if it looks vaguely bell shaped (i'll be most surprised) then you can say work out the standard deviation and the mean.
however, just from your description so far we know you have a lot of infant mortality (that is, when you fix a breakdown you introduce new problems) so a normal distribution is unlikely.
here's an intro to the bathtub curve
thanks greg,
i shall dig into that. there is one thing about these failures that i didn't mention: they are not breakdowns, only failures to operate properly. the equipment works after a reset. so it is more about infrequent disturbances and not about infant mortality.
this situation occurs in many different processes. i have had it in the signalling system for a subway, in a paper machine drive, in a telephone exchange etcetera.
the measures taken are normally filtering, tightening terminals, better grounding and so on. in one case we even changed the cable type. it helped. the problem is that we cannot say with any certainty that it hepled. at least not without waiting three or four (or five?) times the longest interval between earlier failures. and, with three months between episodes, that could mean at least a year's waiting. it would be nice to know when we can send the invoice and also tell the customer that we really helped.
this situation is not very often. most times we find an obvious reason for the problems and then we don't have to wait and see.
the first thing that i would do is try and figure out the exact failure mechanisms.  if you can't recreate the problem, then you will have a very difficult time determining whether you have solved the problem.
agree, melone. but sometimes you are not allowed to do that. a paper machine with usd 10 000+ production per hour is not something you are allowed to stop for investigations.
you have to work indirectly by analysing possible causes and apply probable mitigations and see if it influences the failure rate. it is this latter thing that i would like to find a more scientific approach to. if there is one.
then you have to decide what is more important, fixing the problem or keeping the machine running!  just because you can't take the machine down at a whim, doesn't mean that you can't come up with a logical, practical, creative set of test that can be run at one time, thus maximizing the amount of time the machine "sits idle".
think of it this way, how much are they spending in wasted product and the salary of the people working to solve this problem.  eventually keeping the machine running will be more costly than allowing you to get in their and truly see what's going on.  once you understand the true root cause of the problem, you turn the machine back on, and sleep good at night knowing that you will not run into this problem again.  does management want to solve the problem, or just "band-aid" it?
melone is correct.  sometimes it's hard to explain to someone that "fixing" an infrequent and low-cost problem isn't the right thing to do.  it's also difficult sometimes to get someone to agree to allow you to examine the equipment, because "they have to keep it running."
ultimately, if you want to use statistics based on mtbf, you will have to wait until the next failure occurs, compare the probability of that tbf to the mtbf before you made a change, and see if there is a significant difference.
if you are not in a position to do that, then you have to get more creative.  for example, do a fault tree and reverse fault tree analysis on the equipment to try and figure out everything that could possibly go wrong.  then try to narrow that down to a list of significant factors (by doe, for example).  of those, some can be controlled, and some not.  control the ones you can, and monitor the ones you can't.  then you should be able to at least simulate the effect on performance of changing one of the control factors.  this should let you estimate a new mtbf.  but again, validation will require that you actually measure the results.
hope this helps!
thanks a lot guys!
now, we have that problem again. a lifting bridge that was rebuilt about ten years ago has started to behave erratically. it used to happen once or twice a month. and now it hasn't happened for about a month. we have arranged logging of movements, brakes, motor currents etc on both sides of the bridge. it took several days to do that.
now, the customer says that the bridge "seems to be fixed" and says that no logging is needed any more. we have been told to remove the equipment - without having any results. the bridge operator (the national sea-way authority) understands no electricity and no statistics. nor does the bridge owner (the national road and bridge authority).
we refuse to take the equipment down before we know what is going on. and this is for several reasons:
a) we have invested (or rather the bridge operator has) a lot of money and there is no return on that investment.
b) next time the bridge gets stuck in open position (two national roads going across it) there will be a lot of media coverage - and not very positive...
c) we have a reputation that we get to the bottom of all tasks that we undertake - and we do not want to have that reputation spoiled.
once again: how do you tell a next to ignorant customer about these things? we can not educate him in statistics. is there a way?
$
work out the total cost to society of a bridge malfunction, compare it with the cost of monitoring. that'll give you some sort of break even analysis.
i've just done a course in statistics, sadly it doesn't answer your question, but a weibull analysis is ou main tool for looking at life type data for small numbers of samples.
cheers
greg locock
thanks greg,
i just wrote them a letter regarding cost comparison. i threw in media reactions as well. i think that will start them thinking...