August 24, 2001
Hot-Swappable Power Supply Failures on UPS
Question: Can you help solve the failure of hot-swappable power supplies operating on uninterruptable power supplies (UPS)?
Power supply failures. Over a period of about a month I have been discussing a problem with a reader through email and telephone conversations concerning failure of hot-swappable switching-mode power supply units in equipment at several locations in a large computer facility.
Reader: The failures are independent of power supply manufacturer and type of load (router, server, etc.), but seem to occur in certain locations. At these locations there seems to be oscillations on the interface between the power supplies and the Uninterruptable power supplies (UPS) providing equipment power.
Answer: The first series of exchanges were about filter interactions, including the effects of cables, discussed on my website. Also discussed were possible consultants and others who might help with the problem.
No joy, since after more effort on the readers part I got the email.
Reader: "I am still struggling with the power supply failure problem that I introduce to you about 2 weeks ago and have some new information that may lead a solution." The email went on to provide some data which was discussed and more questions asked.
Answer: I have not been ignoring your email but did not have any immediate answer except the one given below. I have reread your email several times in the interim, but no new ideas occurred.
Reader: At this point I believe that the power supplies are not compatible with the local power distribution units (UPS's) that are installed.
Answer: I know that you have probably done this, but it is the only thing that comes to mind. I would expect that somewhere in the UPS manufacturer's corporate memory they have experienced something similar and may have some solutions, or at least ideas. The problem is, of course, tapping into the corporate memory -- some times it is nearly impossible to get to the person with the knowledge. However, the manufacturer's website mission statement is ... 100 percent customer satisfaction... I would start with the top engineering manager of the product you are using. He/she probably has the title of VP of Engineering, Director of Engineer, etc, or someone on the corporate engineering staff. They probably do not have the answer, but know the engineers in the company most likely to have the answer and may be able to facilitate a contact -- although they try to protect their engineers and have their field people do the support. Some field engineers are very good and they may have the answers, but I suspect you have tried them already with no success. When you have to go into a corporation, I have found that the higher you go, the more courteous the people are and the more helpful information you get. I have no personal knowledge of the manufacturer at this time, but I know many years ago they had some sharp engineering people. That is who you want working your problem, or at least on the team.
Reader: Would it be advisable to remove the capacitors on the power distribution units? I can think of no reason why they need to be there in an environment that is UPS fed and where all of the loads have input filters.
Answer: I'm torn on this one. I'm an experimentalist, so I am always willing to try anything. But I know from experience that without knowledge of, or a model of the system dynamics and what adding or removing capacitance does to the model, you are working totally in the dark. What seems like common sense may be the worse thing you can do when you plug it into the model or system.
Which leads to another thought. Both the UPS and the power supply people should have a simple model of their systems that you can integrate and take a look at. This is something I always try to do and it often leads to the cause of the problem and solution. As R. David Middlebrook says, start with the simplest possible model you can and only add complexity as you need it to explain something that the simple model doesn't. He revived something called the extra-element theorem to make this easier than conventional analysis. The extra-element theorem is described in Erickson, Robert W., and Dragan Maksimovic, "Fundamentals of Power Electronics, Second Edition," Kluwer Academic Publishers, 2001. The examples given are somewhat relevant to your problem.
One of the other people I recommended as a source to query on this problem was Dr. Kasemsan Siri because of talks on similar problems he has given at the IEEE Power Electronics Society Los Angeles Council Chapter. He was kind enough to include me in his replies. They are an order-of-magnitude more professional and useful than my response above. Basically, a plan for taking network analyzer measurements and using the results. He also including pit-falls to be aware of in looking at the dynamics and loops. He also wisely counseled not to touch the system experimentally until you know what is going on from analysis.
One of the things I like about this profession is how helpful people are when you have a problem. And like my experience in going high up in a corporation to get help, often the most knowledgeable, prestigious, and busy technical people in this field are the most generous with their time and help -- if it appears you have given it your best shot and really need some help.
Posted by Jerrold Foutz at August 24, 2001 09:44 AM