LIN_ADVANCE freeze

[nzinov]

I am running d3cb1a8 on delta. When I try to print with LIN_ADVANCE uncommented, printer moves to the start point, then extruder produces some noise for ~1sec and then printing completely stops. Hotend led is blinking at changing rate as usual - so firmware is not crashed at that moment. However, it only outputs busy: processing into console and is not responding to any commands. Any tips on debugging that?

[psavva]

Can you lower your print speed, say to 35mm/s…?

[mottihoresh]

I think the print/move speed may cause it, I am experiencing the same issue. Not sure what's going on. It's not happening with RC8BugFix branch

[Sebastianv650]

@mottihoresh there is no branch "RC8BugFix" - do you mean the code base tagged with "1.1.0-RC8"?

@nzinov can you post the gcode section +- some lines around the start point where it freezes? A test with a low speed as the noted 35mm/s can't hurt also, maybe it helps to find the reason for it.
You might also try RCBugFix at the state without my latest LIN_ADVANCE change.

If somebody knows other issues mentioning such a behaviour, please give me a hint on them.

[mottihoresh]

@Sebastianv650 Meant to write RCBugFix (https://github.com/MarlinFirmware/Marlin/tree/RCBugFix)

I actually spend couple of hours yesterday reading through the issue log. my best guest of what is going on is that the ATMega MCU just chokes since it's trying to process data too fast.

I got non print moves at 160mm/s, and 32 steps per minute, using the new bilinear bed leveling, print speed are at 60 or 80mm/s. When reducing the global feedrate to 50% i noticed no issues, however as soon as I switched it back to 100% the MCU entered panic mode and restarted.

I think this issue may be related:
#4931

My configuration file: https://gist.github.com/mottihoresh/5f51c1d6c07d75c918992125b4e952ea

[nzinov]

@Sebastianv650 here is the begining of my g-code

M104 S235 ; set temperature
G28
G29
M420 S1 Z10
M109 S235 ; wait for temperature to be reached
G21 ; set units to millimeters
G90 ; use absolute coordinates
M82 ; use absolute distances for extrusion
G92 E0
M106 S178.5
;layer 0.4 0
G1 Z0.400 F12000.000
G1 X36.622 Y6.495 F12000.000
G1 X36.622 Y18.274 E0.72007 F1440.000
G1 X35.880 Y21.412 E0.91717
G1 X33.222 Y26.716 E1.27982

I am going to test lowering print speed a bit later. At this moment my issue is not quite like @mottihoresh. He says the MCU is restarted while in my case it apparently is doing a good job of maintaining temperature even after freeze.

[Sebastianv650]

I'm also thinking we have two independent problems here. So your printer is freezing after this line:
G1 X36.622 Y6.495 F12000.000

Then I don't think it's the print speed, F1440 is 24mm/s which is already really low.

I have two problems: No knowledge about Marlin code related to bed leveling (as far as I know, your G29 means you are using it) and no knowledge about delta code handling.

What happens if you try a print without bed leveling, is this possible?

If yes, it would be interesting to know if it works without bed leveling and if yes, does it also work with LIN_ADVANCE enabled. This way it should be possible to get the problematic feature.

[nzinov]

@Sebastianv650 I've tried printing with bed-leveling turned off. Now it makes two circles of skirt with extruder producing strange noise (can't tell for sure but maybe step skip) and then stops in the same manner. Freezes outputting "busy: processing" not responding to commands from host (unfortunately I don't own an LCD)

[psavva]

@nzinov If you turn off LIN_ADVANCE, I guess you don't have the issue? right?
Can you provide you config files in a pastebin, for both Configuration.h, and Configuration_adv.h?
Lastly, If you're using 1/32 step drivers, I suggest to configure your control board to 1/8 or max 1/16 steps, and try the test again.

If lowering the step resolution resolves this issue, then I think it certainly has to do with draining the CPU, and thus failure at some point....

[nzinov]

@psavva Yes, without LIN_ADVANCE I have no issues (multi-hour prints went OK). I use Polulu drivers, so 1/16 steps max. Here lies my configuration: https://gist.github.com/nzinov/ea6544ce3df78a9438c81bf9b1a435a3.

Besides, I really doubt that CPU would fail to calculate moves but still happily chat to console and handle temperature

[Sebastianv650]

I will try to provide you a modified stepper or planner file in the next 1-2 days that outputs some debug data for each move. That should help to find the error.

[nzinov]

@Sebastianv650 OK, looking forward to it.)

[hexane360]

I'm having a very similar problem. Sending M905 K0 fixes the problem. I am also using ABL (linear).

I don't have a LCD, so I'm not quite sure what is happening with the firmware. Help debugging would be appreciated.

Additionally, my extruder motor gets very hot during the resulting fugue state, and sending M112 doesn't seem to have an effect.

[hexane360]

I've been playing around with various things, and the problem seems to be VERY weird.

Some general printer information:
-Lulzbot Mini
-Cartesian
-ABL Linear
-1/8 Microstepping (down from 1/16)
-Latest RCBugFix

And here is the start G-Code that caused the problem:

G21                          ; metric values
G90                          ; absolute positioning
M82                          ; set extruder to absolute mode
;M200 D2.9175 T0 ; set filament diameter
M200 D0 T0; cancel volumetric extrusion
M107                         ; start with the fan off
G92 E0                       ; set extruder position to 0
M140 S55                     ; get bed heating up
G28                               ; home all
M109 S150                     ; set to cleaning temp and wait
G1 Z150 E-10 F75           ; suck up XXmm of filament
M109 S170                     ; heat up rest of way
G12  P1 S2                      ; wipe
G01 Z20 F700              ; raise off wipe pad
M109 S170                    ; set to probing temp
M204 S300                    ; set accel for probing
G29                          ; Probe
M204 S2000                   ; set accel back to normal
G1 X5 Y15 Z10 F5000          ; get out the way
M400                         ; clear buffer
G4 S1                        ; pause
M109 S190    ; set extruder temp and wait
M190 S55 ; set bed temp and wait
G1 Z2 E0 F75                 ; extrude filament back into nozzle
G21 ; set units to millimeters
G90 ; use absolute coordinates
M82 ; use absolute distances for extrusion
G92 E0
;layer 0, z:0.4
G1 E-1.00000 F600.00000
G92 E0
G1 Z0.200 F10800.000
G1 Z0.400 F10800.000
G1 Z0.600 F10800.000
G1 X64.958 Y65.166 F10800.000
G1 Z0.400 F10800.000
G1 E1.00000 F600.00000
M204 S500
G1 F1800

No amount of manually entering commands allowed me to recreate the problem. However, if I started the print with M905 K0 and then paused, entered M905 K25, and resumed, the print had no trouble. Eventually, I started stripping out extruder moves until I got it working. One of the following lines caused it to fail:

G1 Z2 E0 F75
G92 E0
G1 E-1 F600
G1 E1 F600

It seems like the culprit could be one of the following:

• Slow feedrates
• Sign of extrusion (- to + or + to -)
• Something else

Disabling retraction in Slic3r also removes the problem code. However, I don't think that's the root cause.

[hexane360]

@Sebastianv650 I've been poking through planner.cpp getting an idea how LIN_ADVANCE works. The method Planner::_set_position_mm seems to reset the position[] array. However, position_float is only updated after a move has taken place, using memcpy(position_float, target_float, sizeof(position_float));. Is it possible that the first planner move after _set_position_mm is called will use a stale position_float[]?

void Planner::_set_position_mm(const float &a, const float &b, const float &c, const float &e) {
  #if ENABLED(DISTINCT_E_FACTORS)
    #define _EINDEX (E_AXIS + active_extruder)
    last_extruder = active_extruder;
  #else
    #define _EINDEX E_AXIS
  #endif
  long na = position[X_AXIS] = lround(a * axis_steps_per_mm[X_AXIS]),
       nb = position[Y_AXIS] = lround(b * axis_steps_per_mm[Y_AXIS]),
       nc = position[Z_AXIS] = lround(c * axis_steps_per_mm[Z_AXIS]),
       ne = position[E_AXIS] = lround(e * axis_steps_per_mm[_EINDEX]);
  stepper.set_position(na, nb, nc, ne);
  previous_nominal_speed = 0.0; // Resets planner junction speeds. Assumes start from rest.
  ZERO(previous_speed);
}

There may also be other gaps similar to this. Looking through the file, there's a lot of places where position[] is set without position_float[].

[Sebastianv650]

I don't think that's a problem, but I can't say that for 100% because I don't know where all that set_position things are called. According to some comments, they are used for homing, probing and such things. That would mean they will not be active during the main print.
But all freezes I know of happen during a print, not during homing?

I'm still having no idea where this problem comes from. Even with a gcode from another TAZ 5 that produces this issue, I can't reproduce it on my TAZ 5. It seems to be a very specific problem that happens only with specific configs combined with specific numbers in the gcode.
At the moment my only hope is that some day somebody can provide a gcode snipplet that lets me reproduce the error.. :-(

[billyd60]

Sebastian, I have solved most of my issues with RCBUGFIX and it was mostly hardware related. I had a bad enclosure connector on my heated bed cable and the silicone heater was going bad as well. Crazy to have two problems so closely related. Anyway I have a new silicone heater and I am running the cables (power and thermistor for the bed) direct into the RAMBO with no breaks. This is actually amazingly good as I think I was getting some serious losses in power and accuracy with the two standard breaks (Enclosure connector and at the bed) that the Taz 5 has in this line. My bed has never heated so fast or been this stable. So I may have had a balky enclosure connector from day1

Anyway long story short I still get a freeze in printing from time to time (pretty rare though). Other than that lin advance and Rcbugfix are working great and producing the best prints I've ever gotten. The freeze is bizarre though, the printer just sits there, heat still going to the extruder and bed, lcd screen reading normally, just zero movement. Just happened again last night for the first time in weeks, printing in PLA. It seems to happen on large complex models. Maybe just because they take so long. Never have any trouble with small prints.

[Sebastianv650]

Glad to hear that at least some problems were hardware dependent!

When it freezes, is the LCD menu also functional, meaning it's possible to enter and navigate through it? If yes I will continue searching around the stepper ISR handler.

[billyd60]

I had to cycle power to get the extruder off the part (to use the prepare menu move z axis) so the LCD screen was not responsive to the dial control. The extruder and heat bed were still at temperature though, and the LCD screen looked normal in every respect and it was still reading the thermistor temps as well as x y and z coordinates which lined up with where the extruder was approximately (I didn't actually measure just by eye).

[Sebastianv650]

I still have no real answer, but you might try the following with a code that usualy crashes:

Inside void Stepper::ISRhandler(), comment out the lines containing sei() and cli().
Inside void Temperature::isr(), comment out the line containing sei().

This will undo some ISR handling changes I did to Marlin. You might see a lot of serial communication errors this way, so you might print at a lower serial baud rate for this test.

Second option: Undo the changes from above, but add a cli(); just above SBI(TIMSK0, OCIE0B); at the end of void Temperature::isr().

Both changes are "a shot in the dark" if my translator was able to translate what I mean ;-) But at least it's better than nothing..

[billyd60]

I only print from SD card never from USB.

[billyd60]

Sebastian I believe I have isolated the freeze problem. If I stop a print prematurely from the LCD panel and restart the job without cycling power to the RAMBO, the printer will ultimately freeze mid print at some point. It also cause odd sudden drops in the extruder temp and bed temp readings. None of this behavior occurs if I simply start the printer and print a job. Stopping the print from the LCD panel seems to cause a whole host of problems with RCBUGFIX.

All of this is just what I have observed in using the firmware, I can't prove any of it since I can't code.

I hope this puts you on the right path to find out why lin_adv freezes a print job sometimes.

[Sebastianv650]

Thanks, i will try to reproduce that!

[Sebastianv650]

I started a print from SD card, let it run for a minute or so and then aborted it.
Restarted it, let it run for 1-2 minutes. No crash.

How long should it take until the freeze? As there is nothing "counting up" within the stepper or advance code, I have no idea why it should behave different when I let it run longer..
I had a quick look at the issue page (I wasn't activly reading it for some weeks) and there seems to be quite some freezing, temp. jumps and other issues with the recent code page. That make me believe the error might not be in LIN_ADVANCE specific because they are not mentioning it. Maybe advance only increases the probability the error shows up?

[thinkyhead]

I think there's probably some merit to the point that position_float and other additions for LIN_ADVANCE are not being set properly. Note that position itself is never updated until the end of _buffer_line. However the LIN_ADVANCE code updates it right away, at the top of _buffer_line. Likewise the E movement is skipped when in dry-run mode, but this is after the LIN_ADVANCE code is using it. So I think the LIN_ADVANCE code in _buffer_line needs some tweaks to make sure position_float[] has full parity with position[].

[Sebastianv650]

I thought I only doubled the position array by the position_float thing. It looks like I have to dig into the code again and compare what both of them are doing once more.

[Sebastianv650]

@billyd60 Can you try this branch and see if it solves for example your print from SD -> abort & restart problem? It's RCBugFix + a commit that should cleanup all position <> position_float things.
https://github.com/Sebastianv650/Marlin/tree/cleanup-position_float

If it works better now, I would create a PR.

[billyd60]

Hi Sebastian been away from my computer for awhile. Yes I will try it tonight or tomorrow.

Regarding the sudden drops in the measured thermistor temps (after an lcd panel stop print command and restart) it appears as though the code is "losing track" of the actual readings from the thermistors (Bed and extruder) periodically. Ultimately I believe the temperature drops so low that it may trigger cold extrusion protection. This is just a guess though.

Btw this behavior sometimes takes a few hours to show up (both the odd temperature drops and print freeze). So it is quite difficult to reproduce. This suggests a very slow build up of the problem. Perhaps some kind of additive floating point problem that takes tens of thousands of iterations to show up? Again this behavior never happens on my printer without an LCD stop print command and then restart without cycling power. As long as I turn on the printer, and execute a print job without cancelling it, I have never had a print fail. I just completed a 9.5 hour print job with Rcbugfix and it was flawless.

[Sebastianv650]

@billyd60 please read the last posts from issue #5698, this might be another possible reason for the freezes you have.

[billyd60]

Do I need to use your config and adv.h files and change them for the Taz 5 or can I just use my current config and _adv.h files I am currently using for Rcbugfix? I really don't feel like doing all that editing again...

[Sebastianv650]

My two PR are not changing the config files. Depending on how old your current ones are, you might be able to copy and paste them. I would open your current one and the maybe newer RCBugFix one in notepad++ and use the compare function to see if there are structural changes. If not, just use the old ones.

[psavva]

@Sebastianv650, @thinkyhead

I have gone ahead and re-tested it.

No Luck... :(

I have inputted the factor directly in the firmware, and did not use the GCODE.

The first problem I noticed is that the very short moves did not extrude properly.
It was like heavy under extrusion for moves < half a cm.
For some moves which were larger than 1 or 2 cm, seems ok, such as the skirt.

After cranking up the FR to 200%, the Printer froze just like it did before...

Serial Monitor Output is here:
`
start
echo: External Reset
Marlin 1.1.0-RCBugFix

echo: Last Updated: 2016-12-06 12:00 | Author: (none, default config)
Compiled: Mar 5 2017
echo: Free Memory: 3140 PlannerBufferBytes: 1312
echo:V29 stored settings retrieve Y200.00 Zes)
echo:Steps per unit:
echo: M92 X160.00 Y160.00 Z800.00 E385.00
echo:Maximum feedrates (mm/s):
echo: M203 X200.00 Y200.00 Z15.00 E100.00
echo:Maximum Acceleration (mm/s2):
echo: M201 X1500 Y1500 Z500 E5000
echo:Accelerations: P=printing, R=retract and T=travel
echo: Mime (ms), X=max000.00 T1500.00
echo:Advanced variables: S=Min fe=maximu(mm/s), T=Min travel feedrate (mm/s), B=minimum segment time (ms), X=maximum XY jerk (mm/s), Z=maximum Z jerk (mm/s), E=maximum E jerk (mm/s)
echo: M205 S0.00 T0.00 B20000 X9.00 Y9.00 Z0.40 E7.00
echo:Home offset (mm)
echS1 H240 B110 F0
e.00 Z0.00
Auto Bed Leveling:
echo: M420 S0
ech.75
terial heatup parameters:
echo: M145 S0 H190 B50 F0
M145 S1 H240 B110 F0
echo:PID settings:
echo: M301 P51.83 I5.02 D133.75
echo:Filament settings: Disabled
echo: M200 D1.75
echo: M200 D0
echo:Z-Probe Offset (mm):
echo: M851 Z-0.60
echo:SD card ok
echo:SD card ok
echo:enqueueing "M23 wb_dra2.gco"
echo:enqueueing "M24"
echo:Now fresh file: wb_dra2.gco
File opened: wb_dra~2.gco Size: 4023586
File selected
.
.
.
T:189.8 /190.0 @:64 W:3
T:190.0 /190.0 @:61 W:2
T:190.5 /190.0 @:45 W:1
T:190.5 /190.0 @:48 W:0
echo:busy: processing
.
.
.
echo:busy: processing
X:160.00 Y:105.00 Z:10.60 E:0.00 Count X:25600 Y:16800 Z:8480
G29 Auto Bed Leveling
echo:busy: processing
.
.
.
echo:busy: processing
Bilinear Leveling Grid:
0 1 2
0 +0.41 +0.57 +0.63
1 +0.17 -0.31 -0.79
2 +0.49 -0.60 -1.70

X:340.00 Y:290.00 Z:11.86 E:0.00 Count X:54400 Y:46400 Z:8480
X:340.00 Y:290.00 Z:15.00 E:0.00 Count X:54400 Y:46400 Z:8486
X:340.00 Y:290.00 Z:15.00 E:0.00 Count X:54400 Y:46400 Z:8529
echo:busy: processing
echo:busy: processing
`

[Sebastianv650]

Can you copy&paste the line you entered in configuration_adv.h together with the values you used for calculation?
It sounds like an inproper value, I just give it another test with a curvy model using a fixed ratio and even with my 20x magnifier there is no problem with the extrusion.
So it might be a typo, mixup of units or we have another problem here.

[psavva]

#define LIN_ADVANCE

#if ENABLED(LIN_ADVANCE)
#define LIN_ADVANCE_K 300

#define LIN_ADVANCE_E_D_RATIO 0.039912 // The calculated ratio (or 0) according to the formula W * H / ((D / 2) ^ 2 * PI)
// Example: 0.4 * 0.2 / ((1.75 / 2) ^ 2 * PI) = 0.033260135
#endif

I had used the same Values as before, on the same model:
ie: Layer Height 0.24
Filament = 1.75
Nozzle = 0.4mm

GIT Revision Used: (Today 05 March @ 13:00 20addc6 )

Note: Just removed comments to unclog the post

[Sebastianv650]

The ratio is OK. Whatever is causing your under extrusion in detail, this is usualy a symptom of a completely wrong pressure correction. If it's worse than without LIN_ADVANCE enabled at all, the corrections (K and / or the ratio) is much too high. If it's a bit better than without LIN_ADVANCE enabled at all, the corrections is too low.

I guess you did all your K calibration with Cura, meaning K could be off now with the "fixed" workaround.
Can you redo the flat calibration cube, ensuring coasting and simmilar things are disabled. Then put them on a scanner or if not available take top looking down photography with a as good resolution as possible and attach them here.
K=100, 200 and 300 should be a good start set.

[psavva]

I'll give it a go from start to end.

[psavva]

I confirm the issue is resolved.
Thank you @Sebastianv650

[psavva]

@thinkyhead, Please close this issue, as it seems I cannot

[Sebastianv650]

Thanks for your patience @psavva,this issue was a long one!
Do you got another k value now?

[psavva]

Yes, much lower... I'm on 75 now, but am calibrating it further.

[Sebastianv650]

When you found your new k value, it would be good to know if the fixed ratio option is still needed for crash free operation :-)

[LunNova]

Had what seems to be the same issue, even with a fixed E/D ratio set. Freezes, and busy:processing sent forever.

Tried debugging, stepper::isr stops getting called at this point. That would explain why it's frozen, but I don't know why it's not being called.

Not made much more progress than that as getting it to freeze involved doing a dry run of a print and waiting 15 minutes.

[LunNova]

Patch which fixes the freezing but increases the code size by quite a bit. Can probably be made better by someone more familiar with marlin. https://gist.github.com/nallar/d08bd86ec4210ea36d1e4ab824e0e4e5

Problem was that ADV_RATE could return extreme values. 0 would mean advance_isr would be ran every time, and isr would never get called. ADV_NEVER when it shouldn't be would cause advance_isr to not run when it should. Both of these would cause it to stop moving.

[Sebastianv650]

I'm curious, what's your microstepping and step/mm settings?

I don't think ADV_NEVER is a real world problem, but you are right about the 0 result. In fact I spent quite some time thinking about a good solution, without finding one as can be seen in the code.

The problem is, as soon as we get a 0 or another nearly 0 value we are over-utilizing the system. Basicaly we can't do LIN_ADVANCE at this point. Real solutions might be to reduce acceleration or jerk, maybe lower the microstepping. Or find ways to live with lower K values.
When you set it to 1, in fact that means setting it to 16 according to the line NOLESS(OCR1A, TCNT1 + 16);. In reality, this acts as a limiter that prevents LIN_ADVANCE to execute all the calculated needed steps as it was told due to the given acceleration and K value. While it might work (and protect the printer from freezing) I don't like this because the user has no feedback that the system isn't longer working as it's supposed to. Even if the user want's a higher pressure advance and therefore increase K further and further, he will not see his input in the prints. This will lead to confusion.

Stopping the print or shutting down LIN_ADVANCE by setting K=0 might ruin the non-finished print and is therefore also no good solution.

The only real solution I'm aware of is something I can't do with my knowledge at the moment. It would require a rewrite of the planner. Instead of determining a leading axis (axis with most steps to do) and forcing all other axis to follow, we would need to check if the extruder is able to follow with pressure adoption in mind. This would mean to estimate the maximum extruder speed that will occure due to the pressure correction and if it's too high, reduce acceleration until the maximum speed is inside the limits.
This would be the "easy version", as it keeps the constant acceleration Marlin is using. But it might limit the acceleration more than necessary because the maximum extruder speed might be exceeded only during a short time. Therefore, a variable acceleration or at least an acceleration in in for example 3 steps would be better.
In the end, in both cases it's very likely that the needed planner calculations wouldn't be fast enough on our ATMegas..

In the end, if you can test you changes for some time and you are not seeing other problems, it might be the best option we have at the moment. If other people have this issue, please test it too.
I can't help here as I never get a 0 on my system.

[LunNova]

That makes sense. I'm using quite a high steps/mm for the extruder of 912. I should drop it to 16x microstepping.

This issue only occurred occasionally on one part of a print where my slicer made very tight zigzags with lots of small segments. It didn't happen with most prints.

I think for now changing it to return 1 instead of 0 is the least bad option as it prevents a confusing freeze and a ruined print.

[Sebastianv650]

If it's working, you could create a PR for this. There are only some small things that should be changed:

• ADV_RATE should become ADV_Rate because the all capital version is a sign if #define.
• This is something to be answered by the more code style pros, but I guess it should be Stepper::ADV_Rate because this belongs to the stepper class. Then, it would be also possible to write it as Stepper::ADV_Rate(uint16_t t, uint16_t l) as you have direct access to e_steps[], but I guess it's more elegant to have it the way you did.

The code size is only +84 bytes, I think that's no problem!

[Bob-the-Kuhn]

Don't be afraid to drop the microstepping. It doesn't buy you anything as to positional accuracy but it does quiet down the system.

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.