Routing Problems with a 8 bit Microprocessor Board
Marco Ferrigno , 03-01-2022, 02:20 AM
Good morning everyone!
I'm new to this forum. Let me introduce myself. I'm an electronic engineer who works as R&D Engineer for a small company. I'm mainly involved on writing firmware and developing algorithms for microcontrollers. Sometimes I route Pcb Boards for microcontroller applications (by OrCad Pcb Designer).
This time I'm involved in a PCB problem with a microprocessor (an 8 bit old one operating with a crystal of 26.045 MHz). My goal is to find a way to make this specific microprocessor board work. Let me explain better:
bout 2 years ago, in order to save money, my company decided to merge two complementary boards (both 4 layers) - which work fine since more than 20 years - into one. The new merged board was developed by a third part company. It has the same hardware than the two previous boards. After some basic positive tests and putting it in production we received, almost immediately, lots of negative feedbacks from the field. The board suddenly stops working due to unexpected random (and unknown) event. After other specific tests It seems that the bad event is caused by noise coming from outside the board and I'm strongly worried about an EMI problem. Excluding a problem on the firmware (it is the same than the two boards that works well since years) I decided to going back and look at pcb tracks.
This is a 4 layer board and I cannot identify a separation between Ground, Power and signals, maybe because of the high number of signals (address bus, data bus and many I/Os) and high density of components.
I have a lot of question obvously, but
I'm asking if someone can help me to find a book or a good course on the topic so that I can orient myself to route well the board for this specific application (data buses, address buses...)
Watching at some of Robert videos I saw that maybe 4 layers is not a good idea for this board (6 would be a better choice), so I'm deciding to route again the board with 6 layers instead of 4.
a) Can be the following 6 layers order considered a good choice (Rick Hartley on https://www.youtube.com/watch?v=52fxuRGifLU&t=2891s)?
- sgn/Power
- Ground
- sgn/Power
- Ground
- sgn/Power
- Ground
b) I understand that, in this distribution, every time a signal "jumps" from 1) to 5) or viceversa I need to put a Ground Via nearby in order to guarantee a good return path for displacement currents.
Am I right?
Every comment or idea for a better routing is really appreciated
I'll share with you any pcb image if necessary without any problem
P.S.
I know that the processor is an old one, maybe a good 32 microcontroller could do the job, but the company decided to still keep it for a while.
Thank you a lot
Marco
I'm new to this forum. Let me introduce myself. I'm an electronic engineer who works as R&D Engineer for a small company. I'm mainly involved on writing firmware and developing algorithms for microcontrollers. Sometimes I route Pcb Boards for microcontroller applications (by OrCad Pcb Designer).
This time I'm involved in a PCB problem with a microprocessor (an 8 bit old one operating with a crystal of 26.045 MHz). My goal is to find a way to make this specific microprocessor board work. Let me explain better:
bout 2 years ago, in order to save money, my company decided to merge two complementary boards (both 4 layers) - which work fine since more than 20 years - into one. The new merged board was developed by a third part company. It has the same hardware than the two previous boards. After some basic positive tests and putting it in production we received, almost immediately, lots of negative feedbacks from the field. The board suddenly stops working due to unexpected random (and unknown) event. After other specific tests It seems that the bad event is caused by noise coming from outside the board and I'm strongly worried about an EMI problem. Excluding a problem on the firmware (it is the same than the two boards that works well since years) I decided to going back and look at pcb tracks.
This is a 4 layer board and I cannot identify a separation between Ground, Power and signals, maybe because of the high number of signals (address bus, data bus and many I/Os) and high density of components.
I have a lot of question obvously, but
I'm asking if someone can help me to find a book or a good course on the topic so that I can orient myself to route well the board for this specific application (data buses, address buses...)
Watching at some of Robert videos I saw that maybe 4 layers is not a good idea for this board (6 would be a better choice), so I'm deciding to route again the board with 6 layers instead of 4.
a) Can be the following 6 layers order considered a good choice (Rick Hartley on https://www.youtube.com/watch?v=52fxuRGifLU&t=2891s)?
- sgn/Power
- Ground
- sgn/Power
- Ground
- sgn/Power
- Ground
b) I understand that, in this distribution, every time a signal "jumps" from 1) to 5) or viceversa I need to put a Ground Via nearby in order to guarantee a good return path for displacement currents.
Am I right?
Every comment or idea for a better routing is really appreciated
I'll share with you any pcb image if necessary without any problem
P.S.
I know that the processor is an old one, maybe a good 32 microcontroller could do the job, but the company decided to still keep it for a while.
Thank you a lot
Marco
qdrives , 03-01-2022, 02:31 PM
I do not think EMI causes these problems as it generally is rare (for reasonable or better designs), unless they are working in an extreme EMI noisy environment (like welding robot). You could do some tests: https://www.youtube.com/watch?v=CxPy5Ehe0jI and https://www.youtube.com/watch?v=7r81MfHQyXI or https://emc.live/wp-content/uploads/...EMI-30-min.pdf
Same components and same firmware? And still 26.045MHz? With external memory it could be a timing issue. Look at the length of the traces, especially the clocks.
For the stack-up, it is not just the layer count, but the spacing between the copper layers.
Same components and same firmware? And still 26.045MHz? With external memory it could be a timing issue. Look at the length of the traces, especially the clocks.
For the stack-up, it is not just the layer count, but the spacing between the copper layers.
Marco Ferrigno , 03-02-2022, 06:08 AM
Hi qdrives,
Thank you for your reply
I'm not sure if EMI is the real problem. The board is the brain of a lift controller, it is usually in an noisy harsh environment. It also drives some relè and receive some digital inputs from the outside
I'll do some other tests with a rele's coil in order to see if a bad event is catched, very good idea, thank you!
Yes, same components, same firmware and still 26.045MHz
Do you mean that is important to keep the same length of clock and buses being also sure that the memory timinig is ok for this application, right?
Thank you for your reply
I'm not sure if EMI is the real problem. The board is the brain of a lift controller, it is usually in an noisy harsh environment. It also drives some relè and receive some digital inputs from the outside
I'll do some other tests with a rele's coil in order to see if a bad event is catched, very good idea, thank you!
Yes, same components, same firmware and still 26.045MHz
Do you mean that is important to keep the same length of clock and buses being also sure that the memory timinig is ok for this application, right?
qdrives , 03-02-2022, 06:48 AM
One of the questions is perhaps more important: do you filter and handle the inputs correct - both in hardware and in software?
Memory is getting faster. Perhaps if the clock line is short compared to some other signal, the memory does the wrong action. This is not only for clock, but may also do be Rd/Wr and other control lines.
Memory is getting faster. Perhaps if the clock line is short compared to some other signal, the memory does the wrong action. This is not only for clock, but may also do be Rd/Wr and other control lines.
Marco Ferrigno , 03-02-2022, 07:43 AM
About filtering all I/O in hardware and Software I remark that circuits on the new board are the same than the two previous "separate boards", the software is exactly the same....
so maybe I need to compare the traces lengths
I'll check and let you know
so maybe I need to compare the traces lengths
I'll check and let you know
qdrives , 03-02-2022, 04:14 PM
If the old design barely worked correct, the new layout could screw that up to the point that you notice it.
Marco Ferrigno , 03-04-2022, 03:33 AM
Hi,
I Built an electric arc generator. Putting it close to the board It seems that It works fine without problems.... Maybe it is not an EMI problem.
I'm going to compare buses lengths from each other comparing them also with control lines length too
I'm noticing, and I suppose it is not so good, that all buses and relative control lines have a loto of branches in order to reach vaiuous peripherals (ram, flash, PIO, etc...).
I'm also wondering:
With a 26.045MHz Crystal, how long a trace should be in order to avoid reflections phenomenas?
Considering c/f it should be fine because it should be < 11 m. I'm ok with this length for sure.
I Built an electric arc generator. Putting it close to the board It seems that It works fine without problems.... Maybe it is not an EMI problem.
I'm going to compare buses lengths from each other comparing them also with control lines length too
I'm noticing, and I suppose it is not so good, that all buses and relative control lines have a loto of branches in order to reach vaiuous peripherals (ram, flash, PIO, etc...).
I'm also wondering:
With a 26.045MHz Crystal, how long a trace should be in order to avoid reflections phenomenas?
Considering c/f it should be fine because it should be < 11 m. I'm ok with this length for sure.
robertferanec , 03-04-2022, 04:55 AM
What helped me to find problems on several boards is Environmental chamber test. Try to switch on and run the board at -40C Deg - that helps to find interfaces running "on the edge" (they usually stop working or you may noticed errors). But of course, it can be also for example crosstalk or all the kind of other noises created by component placement or routing.
Marco Ferrigno , 03-04-2022, 05:07 AM
I took some measurements on address bus, data bus and control signals...
I think it is not good but fine. I put all data on a chart
I think it is not good but fine. I put all data on a chart
Marco Ferrigno , 03-04-2022, 05:21 AM
Hi Robert!
Thank you for your suggestion. I'll try to find a way to perform a test at -40°C
I also noticed that soldering capacitors very close to the integrated circuits can change the behavior of the board in a worse way (the halt event became very frequent)
Thank you for your suggestion. I'll try to find a way to perform a test at -40°C
I also noticed that soldering capacitors very close to the integrated circuits can change the behavior of the board in a worse way (the halt event became very frequent)
robertferanec , 03-04-2022, 05:25 AM
If you are not sure about memory bus stability, write and run a memory test. There are specific patters you would like to test, you can google for memory test code examples.
qdrives , 03-04-2022, 08:57 AM
@robertferanec makes a good point with the memory testing. Unfortunately, that only works best with RAM memory as Flash is read-only.
For RAM there are many memory test. I have used March C- for my products.
For cold test you could use "freeze spray" - https://nl.farnell.com/c/chemicals-a.../freeze-sprays
For the hot test you could use a hair dryer or heat gun.
About the line lengths - I assume the lengths are in mils? So RD/ is 450mils and Flash CS is less than 50. You might want to increase the length of the control signals (RD, WR and CS) to be relative the longest.
Signals travel at about 150mm/ns (Er~4). Take the speed of the signal, distance difference and reaction time of the memories and you could estimate if this could be a problem.
This does require another layout and production cycle, so do all other tests first!
But you gave one important clue what seems to make it worse - capacitors. The power integrity of this design may not be good.
Are you sure that the voltage regulator is stable?
Some regulators cannot handle ceramic capacitors. Placing more of them (close to ICs) increases the instability.
Are you able to determine if the halt is pure hardware or perhaps a software interrupt (like hard fault) that stops the MCU's normal operation?
Do you have and use a watchdog?
For RAM there are many memory test. I have used March C- for my products.
For cold test you could use "freeze spray" - https://nl.farnell.com/c/chemicals-a.../freeze-sprays
For the hot test you could use a hair dryer or heat gun.
About the line lengths - I assume the lengths are in mils? So RD/ is 450mils and Flash CS is less than 50. You might want to increase the length of the control signals (RD, WR and CS) to be relative the longest.
Signals travel at about 150mm/ns (Er~4). Take the speed of the signal, distance difference and reaction time of the memories and you could estimate if this could be a problem.
This does require another layout and production cycle, so do all other tests first!
But you gave one important clue what seems to make it worse - capacitors. The power integrity of this design may not be good.
Are you sure that the voltage regulator is stable?
Some regulators cannot handle ceramic capacitors. Placing more of them (close to ICs) increases the instability.
Are you able to determine if the halt is pure hardware or perhaps a software interrupt (like hard fault) that stops the MCU's normal operation?
Do you have and use a watchdog?
Marco Ferrigno , 03-04-2022, 09:18 AM
Hi qdrives
The lengths are in mm
The lengths are in mm
qdrives , 03-05-2022, 01:01 PM
450mm for the RD line!? That is something like 3ns.
Data lines vs WR is about 2ns difference.
How about signal integrity?
Why are these traces do long?
Inner layer our outer layer routing?
Data lines vs WR is about 2ns difference.
How about signal integrity?
Why are these traces do long?
Inner layer our outer layer routing?
Marco Ferrigno , 03-07-2022, 03:49 AM
I know, it's a mess
The dimensions of the previous attached graph are the sum of all traces (buses and relative control lines) there are lot of branches in order to reach vaiuous peripherals...
- I ran a simulation with OrCad signal integrity, It seems not very good but it should works (signals rise or fall with some dumped oscillation, but after about 10ns remain stable)
The processor timing for read and write cycle is about 130ns and this can guarantee stable lines on buses and control lines during read and write processes (i guess)
- As I said before, the higher density and the high number of components could be the cause of so long traces. The overall board dimensions are 280 mm x 190 mm
- Routing is mainly on layers top, and inners layers. On bottom layer there is a ground plane (sometime interrupted by pcb traces).
Just to have an idea I attached a picture of the 280 mm x 190 mm board
The dimensions of the previous attached graph are the sum of all traces (buses and relative control lines) there are lot of branches in order to reach vaiuous peripherals...
- I ran a simulation with OrCad signal integrity, It seems not very good but it should works (signals rise or fall with some dumped oscillation, but after about 10ns remain stable)
The processor timing for read and write cycle is about 130ns and this can guarantee stable lines on buses and control lines during read and write processes (i guess)
- As I said before, the higher density and the high number of components could be the cause of so long traces. The overall board dimensions are 280 mm x 190 mm
- Routing is mainly on layers top, and inners layers. On bottom layer there is a ground plane (sometime interrupted by pcb traces).
Just to have an idea I attached a picture of the 280 mm x 190 mm board
qdrives , 03-07-2022, 09:38 AM
A lot of IO extensions? Did I make a bit correct estimation with my figure?
How was this in the old - two board - situation with trace lengths?
How is the stack-up exactly?
Top = signal, Int1 = Gnd plane? Int2 = Signal, Bottom = Gnd+signal?
How was this in the old - two board - situation with trace lengths?
How is the stack-up exactly?
Top = signal, Int1 = Gnd plane? Int2 = Signal, Bottom = Gnd+signal?
Marco Ferrigno , 03-08-2022, 04:25 AM
Yes, You are right!
In the old two boards you can imagine the following:
1) a big I/O board (4 layers - same dimension as the previously attached picture - good ground plane ad a good power plane)
2) a small board (120 mm x 100 mm) with processor, ram flash etc (4layer and good power and ground plane too)
In the new, the stack up is let's say:
Top = Signal
Int1 = Power + Signal
Int2 = Signal + I/O Ground (with "I/O ground" I mean Ground of the first one of the 2 old boards. Are kept separated and joined together in one specific point)
Bottom = Ground
In the old two boards you can imagine the following:
1) a big I/O board (4 layers - same dimension as the previously attached picture - good ground plane ad a good power plane)
2) a small board (120 mm x 100 mm) with processor, ram flash etc (4layer and good power and ground plane too)
In the new, the stack up is let's say:
Top = Signal
Int1 = Power + Signal
Int2 = Signal + I/O Ground (with "I/O ground" I mean Ground of the first one of the 2 old boards. Are kept separated and joined together in one specific point)
Bottom = Ground
qdrives , 03-08-2022, 01:32 PM
The last picture (with the least amount of tracks) clearly has two area's - power and Gnd, or split Gnd, or ???
How I would do the stack-up
Top: signal
Int1: Gnd And do NOT split it! Keep one single Gnd plane
Int2: Signal And Gnd
Bottom: Signal
Whenever (long) traces need to be put on the Gnd layer (int1) have a Gnd trace gap this on the Top layer.
Power is routed, not a plane, nor polygon.
Are the data lines and RD in a bus structure or do you have multiple T splits?
If it is a bus, you might want to see what the effect would be if you add (AC) termination on them.
Alternatively, you could try series termination just above the flash memory (0402 resistor might fit)
How I would do the stack-up
Top: signal
Int1: Gnd And do NOT split it! Keep one single Gnd plane
Int2: Signal And Gnd
Bottom: Signal
Whenever (long) traces need to be put on the Gnd layer (int1) have a Gnd trace gap this on the Top layer.
Power is routed, not a plane, nor polygon.
Are the data lines and RD in a bus structure or do you have multiple T splits?
If it is a bus, you might want to see what the effect would be if you add (AC) termination on them.
Alternatively, you could try series termination just above the flash memory (0402 resistor might fit)
Marco Ferrigno , 03-09-2022, 08:48 AM
- In the last picture (Bottom Layer) there is a split GND
- Data lines and RD have multiple T splits (do you mean R-C series between signals and ground?)
Thank you also for the stack-up suggestion,
Can you help me to figure out when you say: "Whenever (long) traces need to be put on the Gnd layer (int1) have a Gnd trace gap this on the Top layer"?. I don't well understand this point
- Data lines and RD have multiple T splits (do you mean R-C series between signals and ground?)
Thank you also for the stack-up suggestion,
Can you help me to figure out when you say: "Whenever (long) traces need to be put on the Gnd layer (int1) have a Gnd trace gap this on the Top layer"?. I don't well understand this point
qdrives , 03-09-2022, 04:34 PM
Yes, ac termination is R+C. You could also use parallel termination (R only), but that is a high (continues) load. R should the characteristic impedance value. C <= 1n
Are the T splits after the memory?
Termination could be tried before doing a new layout to test if this may be the problem.
How do the signals look on an oscilloscope?
"have a Gnd trace gap this on the Top layer" - see the picture below. The two outer tracks on the top allow for the return signals to stay close the signals in the top.
Are the T splits after the memory?
Termination could be tried before doing a new layout to test if this may be the problem.
How do the signals look on an oscilloscope?
"have a Gnd trace gap this on the Top layer" - see the picture below. The two outer tracks on the top allow for the return signals to stay close the signals in the top.
Marco Ferrigno , 03-10-2022, 03:09 AM
Ok, Thank you. I got it
Yes data lines and RD have T splits after memory, (please see the attached picture where you can see a typical data line path)
I also attached four oscilloscope images where you can see a typical data line and RD\
1) Long time division image with no noise on the bus due to the external rele's coil switch
2) Zoom of 1)
3) Long time division image with some noise due to external rele's coil switch
4) Zoom of 3)
All data lines have 10k pull up resistor (no AC termination nor series termination at the moment)
IMPORTANT
I also noticed that Flash's Chip select has no pull-up resistor while RAMs have (even in the old project), so, two days ago, I connected a 1k pull-up R on Flash's CS. Doing so the board worked for more than 24hours (the best previous time was more or less 8hours)!!!
Yes data lines and RD have T splits after memory, (please see the attached picture where you can see a typical data line path)
I also attached four oscilloscope images where you can see a typical data line and RD\
1) Long time division image with no noise on the bus due to the external rele's coil switch
2) Zoom of 1)
3) Long time division image with some noise due to external rele's coil switch
4) Zoom of 3)
All data lines have 10k pull up resistor (no AC termination nor series termination at the moment)
IMPORTANT
I also noticed that Flash's Chip select has no pull-up resistor while RAMs have (even in the old project), so, two days ago, I connected a 1k pull-up R on Flash's CS. Doing so the board worked for more than 24hours (the best previous time was more or less 8hours)!!!
qdrives , 03-10-2022, 05:02 PM
Both a pull-up or a pull-down is a termination.
I see that the data lines come up a bit slow. I would suggest a lower resistance value, like 4k7 (~1mA) or 2k2 (~2mA) as for the CS.
That noise can definitely cause the problem. I see spikes above 2V which is one of the typical thresholds.
Do you have freewheeling diodes for the relays?
What voltage do you use the for relays?
Good to hear it seems to work better already.
I see that the data lines come up a bit slow. I would suggest a lower resistance value, like 4k7 (~1mA) or 2k2 (~2mA) as for the CS.
That noise can definitely cause the problem. I see spikes above 2V which is one of the typical thresholds.
Do you have freewheeling diodes for the relays?
What voltage do you use the for relays?
Good to hear it seems to work better already.
robertferanec , 03-11-2022, 02:50 AM
- Routing is mainly on layers top, and inners layers. On bottom layer there is a ground plane (sometime interrupted by pcb traces).
PS: just a note, to measure noise on a scope can be tricky. Sometimes what you see on the oscilloscope may not be exactly what is happening on the board e.g. just by touching the track with a probe may change the signal.
Marco Ferrigno , 03-11-2022, 03:10 AM
- On the board every relay has freewheeling diode, but the relay that cause the problem - an external one - doesn't have. External relays not always have diodes in real applications
- All relays are 12V
Connecting all data lines with pull-up terminations it seems getting worst (worst performance ever, less than 1 hour) maybe it depends on where pull-up resistors have their Vcc reference...
Looking at Vcc, it is a Power plane, very fragmented Power Plane...
Regarding Robert's note about oscilloscope, I notice that making a loop with probe an putting it very close to the board (no electricl contact with it) I can see more or less the same noise
- All relays are 12V
Connecting all data lines with pull-up terminations it seems getting worst (worst performance ever, less than 1 hour) maybe it depends on where pull-up resistors have their Vcc reference...
Looking at Vcc, it is a Power plane, very fragmented Power Plane...
Regarding Robert's note about oscilloscope, I notice that making a loop with probe an putting it very close to the board (no electricl contact with it) I can see more or less the same noise
Marco Ferrigno , 03-11-2022, 05:16 AM
I have a question about the new routing
Let's say +12V for relays and +5V for logic sharing same GND as in the picture. I have 2 return path for currents, If I have well understand it a good practice to have a unique Gnd plane. Isn't it?
Let's say +12V for relays and +5V for logic sharing same GND as in the picture. I have 2 return path for currents, If I have well understand it a good practice to have a unique Gnd plane. Isn't it?
qdrives , 03-11-2022, 03:17 PM
Yes, it depends on where the pull-up is placed.
The good thing of "that lowering the pull-up resistance making it worse", is that it is now easier to locate the issue (lets think positive).
Do you use a DIP package relay driver near the bottom left? All the relay drive lines in parallel with the IO extensions. towards the left. I hope you do not mean that the freewheeling diode is in that driver?
"...to have a unique Gnd plane." Unique is not the right word. A single Gnd plane, or a Gnd plane over the entire board and everything is referencing that Gnd plane. Naturally, there may be more Gnd planes over multiple layers, but all connected together multiple times.
The layout picture shows traces going under the relays on the top. Why is that?
Do you know which relays switch other relays? And is it always another relay? If so, you could add an AC snubber ( https://www.homemade-circuits.com/pr...bber-circuits/ )
The good thing of "that lowering the pull-up resistance making it worse", is that it is now easier to locate the issue (lets think positive).
Do you use a DIP package relay driver near the bottom left? All the relay drive lines in parallel with the IO extensions. towards the left. I hope you do not mean that the freewheeling diode is in that driver?
"...to have a unique Gnd plane." Unique is not the right word. A single Gnd plane, or a Gnd plane over the entire board and everything is referencing that Gnd plane. Naturally, there may be more Gnd planes over multiple layers, but all connected together multiple times.
The layout picture shows traces going under the relays on the top. Why is that?
Do you know which relays switch other relays? And is it always another relay? If so, you could add an AC snubber ( https://www.homemade-circuits.com/pr...bber-circuits/ )
robertferanec , 03-12-2022, 02:54 AM
Personally I would recommend to re-route the board. It could help. If you are not sure, maybe try initially a 6 layer PCB, so you can make GND and power planes better + you could reference all your signals to GND (all signals would have GND as a neighbor layer). I would recommend to watch this video: https://youtu.be/52fxuRGifLU
Marco Ferrigno , 03-14-2022, 02:24 AM
Hi everyone and thank you for all suggestions!
Regarding qdrives' questions:
1) Yes in the bottom left side there is a ULN2803A, very popular relay driver
2) Every relay has its own freewheeling diode close to its coil. The one internal to the driver is not connected
Thank you Robert for the linked video. I had already watched but I think I'll rewatch it again.
I think also that, when It is not easy to isolate the problem, maybe there are more than one condition that makes the board not work.
One basic rule I'm learning from you is that every trace needs a close return path. So, with this in mind I'll procede to board re-routing. I'll start from a 6-layer and if possible, I'll go to a 4 layer one
From the linked video I see that a good 6 layer stackUp is
Top = Signal/Power(not plane nor poligon)
Int1 = Gnd
Int2 = Signal/Power(not plane nor poligon)
Int3 = Gnd
Int4 = Signal/Power(not plane nor poligon)
Bottom = Gnd
What do you think about that?
Regarding qdrives' questions:
1) Yes in the bottom left side there is a ULN2803A, very popular relay driver
2) Every relay has its own freewheeling diode close to its coil. The one internal to the driver is not connected
Thank you Robert for the linked video. I had already watched but I think I'll rewatch it again.
I think also that, when It is not easy to isolate the problem, maybe there are more than one condition that makes the board not work.
One basic rule I'm learning from you is that every trace needs a close return path. So, with this in mind I'll procede to board re-routing. I'll start from a 6-layer and if possible, I'll go to a 4 layer one
From the linked video I see that a good 6 layer stackUp is
Top = Signal/Power(not plane nor poligon)
Int1 = Gnd
Int2 = Signal/Power(not plane nor poligon)
Int3 = Gnd
Int4 = Signal/Power(not plane nor poligon)
Bottom = Gnd
What do you think about that?
qdrives , 03-14-2022, 09:58 AM
The "problem" with re-routing the board is that it takes a lot of time to do the routing and production.
Sure, the design will be better and can even be required to continue, but will the root cause be addressed?
Do you know why/where the MCU stops?
Can you locate the source of the noise by creating a loop with the oscilloscope probes (near field probe) like: https://external-content.duckduckgo....6pid%3DApi&f=1
Perhaps you can even fix it and reduce the noise.
A new layout is better, but it takes time.
Another stackup is sig-gnd-sig - sig-gnd-sig, where the distance between layers 1,2 and 3 and 4, 5 and 6 is small (<= 0.2mm) and the distance between layers 3 and 4 is large (>= 0.5mm) This gives you more space for signal traces.
Sure, the design will be better and can even be required to continue, but will the root cause be addressed?
Do you know why/where the MCU stops?
Can you locate the source of the noise by creating a loop with the oscilloscope probes (near field probe) like: https://external-content.duckduckgo....6pid%3DApi&f=1
Perhaps you can even fix it and reduce the noise.
A new layout is better, but it takes time.
Another stackup is sig-gnd-sig - sig-gnd-sig, where the distance between layers 1,2 and 3 and 4, 5 and 6 is small (<= 0.2mm) and the distance between layers 3 and 4 is large (>= 0.5mm) This gives you more space for signal traces.
Marco Ferrigno , 03-14-2022, 10:21 AM
My hope would be to address the root cause before a new routing ...
It would let to use the 100 pcs or more present in stock
MCU is repeating a loop for letting a relay open and close in loop (5s ON and 5s OFF). The relay is external and is located very close to the bottom of the board.
If the noise is somehow inducted from the external relay (aggressor) to the board (victim) maybe an increase of the distance between them would help, right?
If the external relay is removed from its socket the MCU never stops...
It would let to use the 100 pcs or more present in stock
MCU is repeating a loop for letting a relay open and close in loop (5s ON and 5s OFF). The relay is external and is located very close to the bottom of the board.
If the noise is somehow inducted from the external relay (aggressor) to the board (victim) maybe an increase of the distance between them would help, right?
If the external relay is removed from its socket the MCU never stops...
qdrives , 03-15-2022, 10:49 AM
The question is whether the off-board relay has a freewheeling diode? They could be easily added afterwards seeing the socket.
Besides the coil of that relay, what about the load (contacts). Do you see sparks when it opens/closes?
You mention "MCU is repeating a loop for letting a relay open and close in loop..." - what is it waiting for (other than time)? In other words, how/when does it get out of this loop? That input/signal my be to sensitive.
Besides the coil of that relay, what about the load (contacts). Do you see sparks when it opens/closes?
You mention "MCU is repeating a loop for letting a relay open and close in loop..." - what is it waiting for (other than time)? In other words, how/when does it get out of this loop? That input/signal my be to sensitive.
Marco Ferrigno , 03-16-2022, 02:01 AM
About your first question "The question is whether the off-board relay has a freewheeling diode?" I can say that if NOT connected, the board stops after minutes or hours but connecting the freewhwlling diode it stops after days
About your second question "Besides the coil of that relay, what about the load (contacts). Do you see sparks when it opens/closes?"
No if the freewheeling diode is connected, I don't know when it is not connected (I'm testing the board now with the freewheeling diode in order to understand how the position of relay's contacts will be when the board stops... I am waiting since 2 days for the bad event.
About MCU I don't have any access to this firmware, so I need more time to ask to the developer
qdrives , 03-16-2022, 02:44 PM
Actually, I was talking about the relay below the board.
robertferanec , 03-18-2022, 03:28 AM
@Marco Ferrigno I like your proposed stackup. If you can fit it that way, it would be great. We also often use S/G/S/P/G/S with big distance in the middle between S/P.
Hmm, it could be also EMC issue - that is interesting that if you do not use the big relay, it works. It could be also, that tracks on your PCB are picking up the noise from the big relay. I guess in your previous version, the mounting was different (the big relay and the boards were placed differently). Or the noise from the big relay spreads through the circuit.
If you do not mount the boards on the top of each other, will that work?
Hmm, it could be also EMC issue - that is interesting that if you do not use the big relay, it works. It could be also, that tracks on your PCB are picking up the noise from the big relay. I guess in your previous version, the mounting was different (the big relay and the boards were placed differently). Or the noise from the big relay spreads through the circuit.
If you do not mount the boards on the top of each other, will that work?
Marco Ferrigno , 03-18-2022, 05:25 AM
Hi Robert,
I started the routing process with only 4 layers at the moment. The stackup is the one qdrive suggested
Top: signal
Int1: Gnd And do NOT split it! Keep one single Gnd plane,
Int2: Signal And Gnd
Bottom: Signal
I've also in mind the suggestion regarding keeping the return path as close as possible to the traces
If some routing troubles will arise, I'll switch to a 6 layer stackup
Regarding your questions:
- I can fit previous boards version exactly in the same way with the big relay on bottom and It works fine!!! (even if the freewheeling diode on the big relay is not connected)
- On next days I'll try to increase the distance between the new boad and the big relay in order to understand if it works or not
IMPORTANT NOTE:
The new board is working since more than 99 hours only because the big relay has the freewheeling diode on its coil. The Halt event is very unlikely. If I remove the freewheeling diode the board stops within few hours. As said before the big relay is a simulation of an event
I started the routing process with only 4 layers at the moment. The stackup is the one qdrive suggested
Top: signal
Int1: Gnd And do NOT split it! Keep one single Gnd plane,
Int2: Signal And Gnd
Bottom: Signal
I've also in mind the suggestion regarding keeping the return path as close as possible to the traces
If some routing troubles will arise, I'll switch to a 6 layer stackup
Regarding your questions:
- I can fit previous boards version exactly in the same way with the big relay on bottom and It works fine!!! (even if the freewheeling diode on the big relay is not connected)
- On next days I'll try to increase the distance between the new boad and the big relay in order to understand if it works or not
IMPORTANT NOTE:
The new board is working since more than 99 hours only because the big relay has the freewheeling diode on its coil. The Halt event is very unlikely. If I remove the freewheeling diode the board stops within few hours. As said before the big relay is a simulation of an event
qdrives , 03-18-2022, 07:40 AM
@robertferanec and @Marco Ferrigno I think that the missing freewheeling diode is causing problems together with the split Gnd plane.
Do note that is not just the switching noise (radiated emission), but a voltage spike. That spike has to go somewhere.
Also note comment #7: "I Built an electric arc generator. Putting it close to the board It seems that It works fine without problems.... Maybe it is not an EMI problem."
Do note that is not just the switching noise (radiated emission), but a voltage spike. That spike has to go somewhere.
Also note comment #7: "I Built an electric arc generator. Putting it close to the board It seems that It works fine without problems.... Maybe it is not an EMI problem."
Marco Ferrigno , 03-22-2022, 09:45 AM
Hello everyone!
I update you on the latest news.
- With the freewheeling diode (on big relay) the board works since more than a week without any problem
- Increasing the distance between board and big relay of more or less 5 cm. (no freewheeling diode) the problem is back again.
At this point is more likely that it is not EMC problem and qdrives' Hypothesys ( "I think that the missing freewheeling diode is causing problems together with the split Gnd plane" and "That spike has to go somewhere") is stronger than before
Do you know if there is a way I can try to identify the spike's path?
I update you on the latest news.
- With the freewheeling diode (on big relay) the board works since more than a week without any problem
- Increasing the distance between board and big relay of more or less 5 cm. (no freewheeling diode) the problem is back again.
At this point is more likely that it is not EMC problem and qdrives' Hypothesys ( "I think that the missing freewheeling diode is causing problems together with the split Gnd plane" and "That spike has to go somewhere") is stronger than before
Do you know if there is a way I can try to identify the spike's path?
qdrives , 03-22-2022, 02:46 PM
Yes, you can follow the path, but it will be a mix between schematic and physical (layout) and it can be very difficult to determine what the effect will be on the 'next component' as it not be within spec. You would also need to think in magnetic and electric fields. The high dI/dt and dV/dt make all parasitic inductances and capacitances important too.
Perhaps the first place to check: where does the power come from for that relay (coil)? Is that properly filtered?
Perhaps the first place to check: where does the power come from for that relay (coil)? Is that properly filtered?
Marco Ferrigno , 03-29-2022, 07:52 AM
Today I Isolated the big relay. I used dedicated 12V battery to power its coil.
If freewheeling diode is connected the Board works.
If the freewheeling diode is NOT connected then the Board stops as usual. In this case AP1 relay (internal to the Board) is involved.
Can we suppose that sparks on AP1 contact (the orange one on Board) can generate noise going around the board and causing the problem?
If yes, do you know how to limit this noise going around the board?
Consider also that the big relay can be, on a real application, a third parts component in which we don't know in advance if a freewheeling diode is or not connected
The AP1 connections are isolated just in the "apple green" area highlighted on the layout picture
If freewheeling diode is connected the Board works.
If the freewheeling diode is NOT connected then the Board stops as usual. In this case AP1 relay (internal to the Board) is involved.
Can we suppose that sparks on AP1 contact (the orange one on Board) can generate noise going around the board and causing the problem?
If yes, do you know how to limit this noise going around the board?
Consider also that the big relay can be, on a real application, a third parts component in which we don't know in advance if a freewheeling diode is or not connected
The AP1 connections are isolated just in the "apple green" area highlighted on the layout picture
qdrives , 03-29-2022, 09:13 AM
To know exactly where / how the noise causes the problem would require a deep dive in the layout and circuit.
What you do know from this experiment is that the external relay has a huge influence on it.
It (AP1) is not the relay i would have guesses on the layout.
One thing to note is that the layout suggests it is a small signal relay. They are not known for their isolation between coil and contact. Using another relay may also prevent/reduce the issue.
What I mentioned before is the use of an AC termination or RC snubber (same thing). This a bit like a AC freewheeling diode. Not as good, but should work and does not care on polarity. You could add this, regardless of whether a freewheeling diode is installed. I believe there are some formula's, but you could test with a 10nF and 100E.
What you do know from this experiment is that the external relay has a huge influence on it.
It (AP1) is not the relay i would have guesses on the layout.
One thing to note is that the layout suggests it is a small signal relay. They are not known for their isolation between coil and contact. Using another relay may also prevent/reduce the issue.
What I mentioned before is the use of an AC termination or RC snubber (same thing). This a bit like a AC freewheeling diode. Not as good, but should work and does not care on polarity. You could add this, regardless of whether a freewheeling diode is installed. I believe there are some formula's, but you could test with a 10nF and 100E.
Marco Ferrigno , 03-30-2022, 02:03 AM
I don't well understand,
Do you say to connect AC terminations on data bus (close to the flash for example) even if the bus structure has multiple T? Or to connect a snubber on AP1 relay contacts? (The values you are suggesting are more suitable to the second option)
Do you say to connect AC terminations on data bus (close to the flash for example) even if the bus structure has multiple T? Or to connect a snubber on AP1 relay contacts? (The values you are suggesting are more suitable to the second option)
qdrives , 03-30-2022, 08:59 AM
The RC snubber is across the relay contacts.
Marco Ferrigno , 03-31-2022, 01:03 AM
Hi qdrives
I Restored all previous conditions, added an RC snubber on AP1 contact, and the board is working since more than 20 hours
I Restored all previous conditions, added an RC snubber on AP1 contact, and the board is working since more than 20 hours
Marco Ferrigno , 03-31-2022, 10:31 AM
About your question on AP1 relay type, It is a Finder single contact with up to 6kV insulation between Coil and contact. It can carry up to 6 Amps.
Do you think the problem is stil in insulation between coil and contact, aren't 6kV enough?
If no, the layout is very simple, traces goes from contact to the connector on the left... that's all. Do you think a good ground plane with vias around the contact can improove the behavior for the new layout?
Do you think the problem is stil in insulation between coil and contact, aren't 6kV enough?
If no, the layout is very simple, traces goes from contact to the connector on the left... that's all. Do you think a good ground plane with vias around the contact can improove the behavior for the new layout?
qdrives , 04-01-2022, 09:34 AM
Hmmm... 8mm. Makes you wonder how it is possible.
What about the "clattering"? relay test that Kenneth Wyatt showed? Also have the cables near the existing wiring and traces.
What about the "clattering"? relay test that Kenneth Wyatt showed? Also have the cables near the existing wiring and traces.
Marco Ferrigno , 04-01-2022, 10:02 AM
What is your opinion, can you help me to better understand? Do you think the problem is in the coil and contact insulation?
Anyway I'll Do Kenneth Wyatt's relay test again and I'll get back to you.
Anyway I'll Do Kenneth Wyatt's relay test again and I'll get back to you.
qdrives , 04-02-2022, 04:07 PM
That is why I wrote "Makes you wonder how it is possible." With 8mm between contacts and coil the problem may be somewhere else, like the relays next to AP1.
However, coming to this point:
1) Current boards. You have located one source that triggers the issue and also have a work-around for it. Either freewheeling diode or RC snubber, with the last being a more universal solution not having a polarity. This can also be added to existing boards.
2) New design. The split Gnd may additionally make the board more sensitive. A new layout is needed. Add the solution for the relay too!
3) Theory and better understanding. With the clattering relay test you may be able to detect if and where there is a radiated immunity issue. I think that for any of us here on the forum we could only assist further by taking a deep dive in schematic and layout. As that would be publicizing your intellectual property, I think that you are on your own. Finding the exact "path" of the spike to the MCU failing is very educational and satisfying, but it may take a lot of time.
In short, if the clattering relay does not show an additional issue, you could very will only do steps 1 and 2 (that is, not finding the "path"). I know my manager never wanted me to go beyond finding the quick fix. I do (and did) and it helped me a lot gaining knowledge for the next design.
However, coming to this point:
1) Current boards. You have located one source that triggers the issue and also have a work-around for it. Either freewheeling diode or RC snubber, with the last being a more universal solution not having a polarity. This can also be added to existing boards.
2) New design. The split Gnd may additionally make the board more sensitive. A new layout is needed. Add the solution for the relay too!
3) Theory and better understanding. With the clattering relay test you may be able to detect if and where there is a radiated immunity issue. I think that for any of us here on the forum we could only assist further by taking a deep dive in schematic and layout. As that would be publicizing your intellectual property, I think that you are on your own. Finding the exact "path" of the spike to the MCU failing is very educational and satisfying, but it may take a lot of time.
In short, if the clattering relay does not show an additional issue, you could very will only do steps 1 and 2 (that is, not finding the "path"). I know my manager never wanted me to go beyond finding the quick fix. I do (and did) and it helped me a lot gaining knowledge for the next design.
Marco Ferrigno , 04-04-2022, 01:36 AM
Hi qdrives,
Thank you a lot for your help,
Yes, I performed the Kenneth Wyatt's clattering relay test, going around the board without any failure detection.
You're also right saying that my manager wants now to apply a solution to the existing boards. A test on the field is going to be the next step. At the same time I'm routing the board keeping in mind your suggestions. The stackUp is the one you suggested on #18, Also your suggestion on comment 20 is in my mind.
I think a good choice would be to have the ground plane extended under relays (contrary to the existing layout).
About the relay side of the new board I'm wondering if multiple ground planes filled with vias that continuosly connecting gnd layers is a good think or not
Thank you a lot for your help,
Yes, I performed the Kenneth Wyatt's clattering relay test, going around the board without any failure detection.
You're also right saying that my manager wants now to apply a solution to the existing boards. A test on the field is going to be the next step. At the same time I'm routing the board keeping in mind your suggestions. The stackUp is the one you suggested on #18, Also your suggestion on comment 20 is in my mind.
I think a good choice would be to have the ground plane extended under relays (contrary to the existing layout).
About the relay side of the new board I'm wondering if multiple ground planes filled with vias that continuosly connecting gnd layers is a good think or not
qdrives , 04-04-2022, 02:49 PM
Yes, I think it is best to have the Gnd plane cover the entire board.
In the screenshot you have traces running to the relay driver. Could you move them to not be between the coils and contacts?
Perhaps even a polygon pour on the top below the relays and many via's to connect them to the Gnd plane as a Gnd fence between the coils and contacts
I do not know if placing via's next to contact (to connector) traces would be much help.
In the screenshot you have traces running to the relay driver. Could you move them to not be between the coils and contacts?
Perhaps even a polygon pour on the top below the relays and many via's to connect them to the Gnd plane as a Gnd fence between the coils and contacts
I do not know if placing via's next to contact (to connector) traces would be much help.
Marco Ferrigno , 04-06-2022, 05:12 AM
Do you say as in the picture?
Do you also think multiple copper poor in other layers under relays is a good practice too?
Do you also think multiple copper poor in other layers under relays is a good practice too?
qdrives , 04-06-2022, 07:58 AM
Do I understand that you change the AP1 relay? I expected it to be problematic, but looking at the datasheet you provided, I had to reconsider and no longer have that feeling that it is the cause.
Anyhow, looking at my colored capture below...
1) The relay in question is AP1, right? Marked in yellow.
2) Why is the trace in green running parallel to the contact traces of AP1?
3) Are the components in the red area on the top or bottom?
As to the division and via fence - Yes exactly, much better.
As to your question regarding polygon pour: the relays are on the top, have the contact traces run as much on the top too. A polygon pour between those (noisy) signals and the rest of your signals.
Anyhow, looking at my colored capture below...
1) The relay in question is AP1, right? Marked in yellow.
2) Why is the trace in green running parallel to the contact traces of AP1?
3) Are the components in the red area on the top or bottom?
As to the division and via fence - Yes exactly, much better.
As to your question regarding polygon pour: the relays are on the top, have the contact traces run as much on the top too. A polygon pour between those (noisy) signals and the rest of your signals.
Marco Ferrigno , 04-06-2022, 08:31 AM
Yes I chenged AP1 and AP2 relays anyway. They now are the same than others (less variety on BOM is a good pratice too)
The relay you marked in yellow (where there is the green trace) is another one, it's name is P
About your question 2) "Why is the trace in green running parallel to the contact traces of AP1?" .. I need a signal trace to go from coil to contact on P relay. Do you suggest any different path?
Question 3) All components marked in red are on top side
Contact traces are overlapped between top and Int2. There is GND in the middle (on int1) while the trace you highlighted in green is on Bottom layer
I'm not sure about your questions, so I attached all layers in order to give a better idea of what I'm doing
Last thing, shoud I insert RC snubbers anyway between relays contacts? Or GND poor and vias are enough?
The relay you marked in yellow (where there is the green trace) is another one, it's name is P
About your question 2) "Why is the trace in green running parallel to the contact traces of AP1?" .. I need a signal trace to go from coil to contact on P relay. Do you suggest any different path?
Question 3) All components marked in red are on top side
Contact traces are overlapped between top and Int2. There is GND in the middle (on int1) while the trace you highlighted in green is on Bottom layer
I'm not sure about your questions, so I attached all layers in order to give a better idea of what I'm doing
Last thing, shoud I insert RC snubbers anyway between relays contacts? Or GND poor and vias are enough?
qdrives , 04-07-2022, 09:32 AM
Starting with the last - Yes, I would add the RC snubber as it has shown to prevent the issue so far. As mentioned, an RC snubber does not have a polarity and can be used across relay contacts too.
In the original layout, you did not have a Gnd (polygon) between the connector for AP1 and the SMD components directly above it. Perhaps there the spike crosses and causes the system to hang. Although the distance between the pins is less...
Could you move the connector down a bit, increasing the distance between the connector and the components. Perhaps centered with the connector below it.
Looking at the Gnd polygon pour on the top, you are creating antenna's. Either remove them or add via's so that it is no longer an antenna.
In the original layout, you did not have a Gnd (polygon) between the connector for AP1 and the SMD components directly above it. Perhaps there the spike crosses and causes the system to hang. Although the distance between the pins is less...
Could you move the connector down a bit, increasing the distance between the connector and the components. Perhaps centered with the connector below it.
Looking at the Gnd polygon pour on the top, you are creating antenna's. Either remove them or add via's so that it is no longer an antenna.
Marco Ferrigno , 04-21-2022, 03:16 AM
Hi everyone,
Thank you again for all your suggestions.
The new routing is almost done!
On all layers but TOP, I used full GND copper poor connected by lots of vias in order to be sure of good gnd connections between layers (it should increase noise immunity). Anyway I attached screenshots of each layer and overall 3D rendering.
What do you think about that? Any further suggestions?
Thanks again
Thank you again for all your suggestions.
The new routing is almost done!
On all layers but TOP, I used full GND copper poor connected by lots of vias in order to be sure of good gnd connections between layers (it should increase noise immunity). Anyway I attached screenshots of each layer and overall 3D rendering.
What do you think about that? Any further suggestions?
Thanks again
qdrives , 04-21-2022, 09:25 AM
Now that is a good ground plane (Int1).
Use our interactive Discord forum to reply or ask new questions.