Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Looking to collaborate, have some questions #1

Open
mcheah opened this issue Nov 4, 2018 · 170 comments
Open

Looking to collaborate, have some questions #1

mcheah opened this issue Nov 4, 2018 · 170 comments

Comments

@mcheah
Copy link

mcheah commented Nov 4, 2018

Hey, thanks for posting the pinout, this saves me a whole bunch of work.
I developed an STM32-based Marlin port for the Monoprice Mini Delta and just got a Lerdge board as it's much higher performance. Porting should be pretty easy to do, but I have some questions about the system that I'm hoping you can answer. Specifically I'd like to know how you bypassed the write protect without clearing the flash rom, and how you were able to probe the GPIO pin configuration registers.
Hit me up on reddit it'd be nice to work with somebody else on this.

@LoialOtter
Copy link
Owner

Very welcome. The board is really quite impressive for being so cheap.

The readout protection on the STM32 line is pretty good (I've developed products that made use of it) but there are some interesting issues with it too. One is that protection of RAM and peripherals is optional. I just and hooked up to JTAG and read from those locations on RAM. Note that it's very important that you do it carefully so you don't accidentally read from any location in flash as that will lock down the JTAG access and you'll no longer be able to read from anywhere. I was really not expecting that to work so I had also pulled out my multimeter to do things the old fashioned way.

Once I had the map done there's a way to erase the entire flash by disabling the readout protection. This then leaves you with a fully unlocked but fully erased device.

@mcheah
Copy link
Author

mcheah commented Nov 4, 2018

Yeah, i did something similar on the stm32F0 firmware on the monoprice printer. I see that both the bootloader and the application bin files are available, so I'm tempted to just blow it all away and start from scratch, but I want to make sure i have the memory map right first. I'm assuming the bootloader ends at 64k, do you know anything about that? Also do you have any info on the LCD / touch screen they used? Tbh I'll likely never implement since the goal is to run off of octoprint. Also do you know why they have so many frikkin mosfets all over the board? Even for the stepper drivers. I'd understand for the heater/fan but it just seems excessive for digital inputs. Maybe they need to level shift to 5V?

@LoialOtter
Copy link
Owner

The mosfets are proper driving and separation so any current spikes will never hit the CPU. They are proper mosfet drivers so also do level translation IIRC. Honestly the IO is probably the best designed I've ever seen in a reprap community driver board. Everything's been given proper inputs and outputs.

My only real complaint from a 3d printing side is that the heated bed driver isn't on the board so you'll need to add a SSR or mosfet or something for that.

The LCD is being driven by one of the DMAs rather than any special peripheral. This means they have a framebuffer in ram that they're simply drawing to and the DMA just copies that onto the display. The pinout should be pretty obvious. It's just a standard 8080/6800 LCD and there's specs online for the resistive touchpad chip on there; I didn't get too deep into it as I was going to go deal with that later and then got distracted from the project.

As for the bootloader and flash layout, no idea as I'm replacing the whole thing. Don't forget the CPU itself has DFU so you don't need anything special, just pull a pin low when you power it on and it'll come up in windows/linux and you can program - no special bootloader needed.

@mcheah
Copy link
Author

mcheah commented Nov 5, 2018

I figured they were kind of superfluous since most stepper drivers support CMOS inputs, but I suppose in the case of a short or failure with the high voltage supplies going to them you'd want to protect the CPU. Just a lot higher component count than I'd expect from a board this size and cost. Agreed on the bed driver. My suspicion is that they realized that it'd be difficult route that much power over a small 2-pin connector and opted to have it supplied externally. While that's definitely the more flexible route, it's frustrating because the provided solution is a dongle to a dongle to supply a heatbed and potentially requires a second power supply or a jury rigged connection.

Thanks for the info on the LCD. I have no idea how to look up info on these screens, most the time they come with very few markings and a gigantic parallel bus. It'd be nice if somebody made a nice library for these kind of things. IMO the graphics and UI design for a product like this is probably 3x the work of the motor control part. I'll likely ditch the whole thing and hook up a simple reprap display through the expansion connector.

Yeah, I've got a ST-Link hooked up through the SWD pins already, I need the ability to debug code directly so I'm skipping the DFU route. What are your plans for the firmware? Are you looking to adapt Marlin/RepRap/Klipper/Smoothie to Lerdge? Or build your own from scratch?
The other thing I didn't like about the board is that there's no native USB port for communication, you have a buy an adapter with a CH401 chip. I tried plugging in a USB A<->A M/M but nothing showed up, I think they have it hooked up as host only.

I'd like to retain the ability to run the stock firmware so I can go back and forth to compare functionality. This is how I was able to debug a 5 degree offset on the temperature readings on my Marlin project.

@LoialOtter
Copy link
Owner

@LoialOtter
Copy link
Owner

M/M works fine. Not standard but hey, you just have to configure the system to not use OTG and instead force device (so a UI is needed if you want it to do host).

The DFU is there as a bootloader for users. I haven't tried erasing a chip using it though that would be interesting. It allows you to ignore their bootloader and just use that if you wish or make one yourself. I've made a few in the past based on ChibiOS which works great on this line of chips.

@mcheah
Copy link
Author

mcheah commented Nov 5, 2018

Yeah, I'll get it working with M/M, but it's not ideal. I don't mind losing the USB key, I don't quite see the need with an available SD slot.
DFU you still need to hook up to a PC and run some STM tool for flashing right? It just runs some hard-coded bootloader in ROM if I recall, I don't think you can put in a custom routine to load off a USB flash disk and copy over firmware.bin for example. So in that scenario you'd require the user to buy a USB M/M cable just for flashing, and I'd just as re-use their own bootloader at that point.

@LoialOtter
Copy link
Owner

DFU under linux is just a normal package - it's not just ST as well as DFU is a a standard for uploading firmware to USB devices.

For me USB is pretty important as it gives the command prompt for python. My plan is to make a lowlevel driver set which will get the steppers and temperature control reliably running then use python to do the motion planning and g-code to control interface. I want to go this route mostly due to wanting to have a platform that's super easy to get working with unusual hardware. As an added benefit, micropython also supports SD cards and can load it's code off there or the 4MB flash that's also on the board.

@xC0000005
Copy link

Does anyone have a lerdge-x board they've already reset the bootloader on? I think I've managed to fully decrypt the existing one (and I'm fairly certain the firmware's completely decrypted now), but obviously once I flip the bit to reset read protect and rewrite, I lose my encryption breaking ability

@mcheah
Copy link
Author

mcheah commented Dec 5, 2018

They bothered encrypting the firmware? That's a next step. I'm planning on running third party firmware anyway if no one else steps up i can take the risk.
Is the bootloader encrypted too?

@mcheah
Copy link
Author

mcheah commented Dec 5, 2018

If you can decrypt the firmware, then the easier test may be to encrypt your own firmware. Write a blinky program, encrypt and upload it and if it works then you know you've got the right encryption scheme. That way you don't have to touch the bootloader in case something goes wrong. You can then read out the bootloader bit by bit and compare against the bootloader binaries.

@xC0000005
Copy link

xC0000005 commented Dec 5, 2018 via email

@mcheah
Copy link
Author

mcheah commented Dec 5, 2018

Very interesting, I will read through all the details when I have more time. I had briefly tried altering the image files by appending zero's the the end of all the binaries. It seemed to me that they didn't do a checksum or anything since the modified firmwares loaded just fine. I was hoping to crash the processor by causing it to write beyond the flash ROM memory space (and thus determine the firmware memory address), but it seems they truncate or ignore the extended image.

@xC0000005
Copy link

xC0000005 commented Dec 5, 2018 via email

@LoialOtter
Copy link
Owner

As soon as you're able to make a payload it'll be really easy to dump the whole image. Code running from flash has 100% access to flash for obvious reasons.

I have one board that I haven't erased and a second one I have - let me know what's needed to test.

@xC0000005
Copy link

xC0000005 commented Dec 6, 2018 via email

@xC0000005
Copy link

decrytpedbootloader.bin.zip
Let's try this again. Now you should get it.

@xC0000005
Copy link

After checking again, the decrypted bootloader definitely starts at 0x10000, so there's something earlier, I just don't know what.

@LoialOtter
Copy link
Owner

the ST parts have an interesting problem when it comes to bootloaders - the first 4k is not rewritable if readout protection is enabled. This problem generally means a dual bootloader setup. The first bootloader may be the same but compiled to operate in this range (and they may have found out about this after shipping like I did).

My solution when I found out about this was to replace a call that happened outside this 4k limit in the original bootloader with a reconfigure and restart from a 4k image. All later releases of that hardware I instead included a 4k first boot that simply reconfigured the chip for operation at 4k and restarted there.

@xC0000005
Copy link

xC0000005 commented Dec 7, 2018 via email

@mcheah
Copy link
Author

mcheah commented Dec 7, 2018

I think I've got a project written for a similar STM32F4 processor to do the memory readout if that's helpful here. I think it should be pretty easy to port over to F407 from the F446. You'll just need to expose one of the UART interfaces. The F07 projects are the cortex M0 version I adapted for the MPMD.

https://github.com/mcheah/stm32MemStream/tree/master/stm32MemStream

You'll need systemworkbench for stm32, and will need to update the pinmappings, but might give you a head start. I might have some time this weekend to give it a try myself now that we know the method to encrypt/load arbitrary code. There's a project called uBlinky that's a barebones blinky program as well.

FYI, I believe the UART/SWD pins are brought out to the 1mm header pins next to the USB port, so you should be able to do most of this with no soldering.

@xC0000005
Copy link

xC0000005 commented Dec 7, 2018 via email

@mcheah
Copy link
Author

mcheah commented Dec 7, 2018

Easiest would be to just disable interrupts, no?

@xC0000005
Copy link

xC0000005 commented Dec 7, 2018 via email

@xC0000005
Copy link

Sort of basic question - which LEDs are PC7 and PC6 connected to? And is it writing high to them that turns them on or low?

@xC0000005
Copy link

Some updates (this isn't what I was looking for, but the things you find when you go looking...):
The LCD is addressed in FSMC mode. The control register is at 0x60000000, the data register at 0x60020000.
screen shot 2018-12-08 at 8 56 38 am
I can't tell what the controller is based on the init sequence, but if nothing else, you can simply string the commands together and it will work.

I also attached a list of the pins that are configured for FMSC (that's the alternate function 0xC).
screen shot 2018-12-07 at 2 41 13 pm

@xC0000005
Copy link

Also, I have zero idea now where anything is loading. I have the DLion source now thanks to someone on hackaday - DLion is the ancester of the lerdge board - none of the encryption, but you can see the structure and nod and say "I can see how they got here from there." Particularly when you look at the font update/SPI Flash code it should be familiiar.

The code is built with Keil, and Keil has a habit of sticking the vector table near the end. If I load the firmware at 0x10000, the vector tables correctly resolve to a set of infinite loops (which is usually the best way to track down an unknown arm vector table). The problem is, they also make some sense at 0x60000. The bootloader does NOT at 0x0000.

I'm thinking my goal here is shifting - this bootloader is a pain in the ass to work with. What I want to do is get in and dump out the base image so people can DFU back, then we'll drop something sane on for everything else. Also, once we have the core image, it should be possible to disable flash protect and set it up. Warning - the bootloader will re-enable flash protection if it's disabled, during the update sequence.

@mcheah
Copy link
Author

mcheah commented Dec 8, 2018

If you can isolate where the bootloader is setting the security bits, presumably you could edit the binary to NOT that no? Or take a different route and just provide your own bootloader instead of the stock one.

What program are you using to disassemble the decrypted binary? How are you getting the symbols for the function names? Is that from the DLion source?

@carl1961
Copy link

carl1961 commented Dec 8, 2018

Here is a Dlion manual, I downloaded before the site was took down, on page 24 starts where is shows how to burn the boot-loader, maybe it has some clues for you. I am so new to this so as of right now I have no clue what needs to be.
Dillon motherboard manual-ProV3.2 (1).pdf

also Dlion Marlin GitHub of a older version may be useless

https://github.com/Pomise/Dlion-Stm32-3D

@xC0000005
Copy link

Program I'm using - BinaryNinja. Function names were added by myself, and the name is almost always evolving. Delay was 'Mess with Sys Clock." Followed by "Wait for SysClock" and then when I looked at how it was used, "Delay." The LCD code, I looked at ST's app note on interfacing with TFT LCDs and saw it used the FSMC and said "Hey, that looks familiar." If anyone has a board which they can load code on (erased), check the device ID - that will probably give us a clue to the controller for the LCD. Yes, I could disable the function to re-enable protection. It's interesting that they thought of so many contingencies.

@mcheah
Copy link
Author

mcheah commented Jan 10, 2019

Interesting. I was gonna say, full FAT seems overkill

@xC0000005
Copy link

xC0000005 commented Jan 10, 2019 via email

@mcheah
Copy link
Author

mcheah commented Jan 10, 2019

I have not,
my firmware saves settings to the SD card anyway. I'm curious why they need an I2C flash if they can just save to the internal flash. Maybe endurance concerns? Maybe it has something to do with the power loss recovery. Does Marlin support power loss recovery? Or was that an add-on prusa made for their fork?

@xC0000005
Copy link

xC0000005 commented Jan 10, 2019 via email

@mcheah
Copy link
Author

mcheah commented Jan 10, 2019

makes sense. I can try pulling data off the I2C flash if you want to focus on the NOR one

@xC0000005
Copy link

xC0000005 commented Jan 10, 2019 via email

@mcheah
Copy link
Author

mcheah commented Jan 10, 2019

I was able to initialize the I2C driver and read the contents of the serial FRAM. I've attached the dump. It uses standard EEPROM command format <I2CAddr+RWbit> A15:8 A7:0 D7:0. I used the HAL_I2C_Mem_Read/HAL_I2C_Mem_Write functions, library like extEEPROM should work fine. See Here for the test code

I2C address is 0xA0 (including R/W LSB)

EEPROM_dump.txt

@xC0000005
Copy link

xC0000005 commented Jan 10, 2019 via email

@xC0000005
Copy link

xC0000005 commented Jan 11, 2019 via email

@jmz52
Copy link

jmz52 commented Jan 11, 2019

@xC0000005 I've reworked TFT and touch code in https://github.com/jmz52/Marlin/tree/MKS-Robin-Touch-POC
During initialization "com" function exposes IO function to "dev" function via
struct LCD_IO {
uint32_t id;
void (*writeRegister)(uint16_t reg);
uint16_t (*readData)(void);
void (*writeData)(uint16_t data);
void (*writeMultiple)(uint16_t data, uint32_t count);
void (*writeSequence)(uint16_t *data, uint16_t length);
void (*setWindow)(uint16_t Xmin, uint16_t Ymin, uint16_t Xmax, uint16_t Ymax);
};

These IO functions are used directly by "dev" to overcome limitations of u8g and to get significant performance boost.
Initialization sequences were reworked to support 16-bit data required for ili9328 controller.
Because of these 16-bit data requirements there is no backward compatibility with original u8g 'com' functions calls.

From you point of view what is the preferable design for 'com' function - keep IO structure to overcome u8g limitation or recreate 'com' function from scratch and use completely new data, methods, parameters and data formats?
Example:
u8g 'com' flow: set 'register select' line; write byte; reset 'register select' line; write byte
new 'com' flow: write register; write data

Update: Also touch screen calibration was reworked to support MKS Robin TFT v1.1
Touch calibration tool for Robin's 320x240 screen - https://github.com/jmz52/MKS-Robin-toolkit/tree/master/lcd_and_touch
Some bits of it might be useful for you too.

@xC0000005
Copy link

xC0000005 commented Jan 11, 2019 via email

@jmz52
Copy link

jmz52 commented Jan 11, 2019

SPI vs. FSMC can be done with two extra calls like 'begin' and 'end' that will be empty for FSMC and set/reset CS pin for SPI. A0 a.k.a Register Select pin manipulation fits perfectly to writeRegister and writeData calls. And the 'reset' call is universal.

My only concern here is 16-bits vs 8-bits data over SPI. There are few ways to deal with this problem,
but SPI version of TFT screen is needed to test exact controller's behavior.

@mcheah
Copy link
Author

mcheah commented Jan 12, 2019

For ESP8266 there's goofy stuff like 9-bit SPI for 18-bit color mode too. Which begs the question, how are the different color formats being handled? Is this something that can be abstracted or does that live at another level?

@xC0000005
Copy link

xC0000005 commented Jan 12, 2019 via email

@jmz52
Copy link

jmz52 commented Jan 14, 2019

@xC0000005, I think I've found the math behind decryptionTable.
Two function below provides the same results as 'encryptionTable.get()' and 'decryptionTable.get()'
def encryptByte(byte):
byte = 0xFF & ((byte << 6) | (byte >> 2))
i = 0x58 + byte
j = 0x05 + byte + (i >> 8)
byte = (0xF8 & i) | (0x07 & j)
return byte

def decryptByte(byte):
i = 0xA8 + byte
j = 0x02 + byte + (i >> 8)
byte = (0xF8 & i) | (0x07 & j)
byte = 0xFF & ((byte >> 6) | (byte << 2))
return byte

@xC0000005
Copy link

xC0000005 commented Jan 14, 2019 via email

@jmz52
Copy link

jmz52 commented Jan 14, 2019

I've looked for patterns in binary numbers format of decryptionTable, to find what bits of value are flipped by each bit of key (first guess was that there is a set of XOR operations on 0xAA byte). This gave me a set of 8 rules that successfully decoded decent chunk of decryptionTable. Then I've tried to create finer rules to cover remaining values and found that XOR is not the right tool to use (rules became more complex than patterns visible in HEX view).
So I've created 8 maps - one for each bit of decryptionTable's values to look at these pattern in HEX view. These maps turn out to be the key (example: bit 0x04 is set for odd keys when key is less than 0x58 and for even keys when key is from 0x58 to 0xFF).
While working with these maps I've found that starting point is 0x58 and that lower three bits in key have offsets shifted at 0x58.
First working version had 8 rules - one for each bit.
for i in range(256):
if i < 0x58:
j = i + 0xA8
k = j + 2
else:
j = i - 0x58
k = j + 3
a = 0x00
a |= 0x01 if j & 0x40 else 0
a |= 0x02 if j & 0x80 else 0
a |= 0x04 if k & 0x01 else 0
a |= 0x08 if k & 0x02 else 0
a |= 0x10 if k & 0x04 else 0
a |= 0x20 if j & 0x08 else 0
a |= 0x40 if j & 0x10 else 0
a |= 0x80 if j & 0x20 else 0

Then I've reverted these rules to create encryption function with another 8 rules and reworked code to make it as short as possible. Never expected to end with just 4 lines though.

@xC0000005
Copy link

I don't know if @mcheah already found this but I'd bet so: The PDF has the step and dir pins for the extruder on the lerdge x reversed, as I found out today while attempting a test print. (On the other hand, hooray for finally getting to the point of a test print).

@mcheah
Copy link
Author

mcheah commented Jan 22, 2019

You know, I noticed that at one point, but I assumed I had fat-fingered it into the header file. Good to know I'm not crazy

@xC0000005
Copy link

@LoialOtter, I have a working extruder with these pins:
#define E0_STEP_PIN PA4
#define E0_DIR_PIN PA3
#define E0_ENABLE_PIN PA5

@Dzenik
Copy link

Dzenik commented Jan 25, 2019

Help guys figure it out. I tried all the ones. Simple blink code does not work.
PlatformIO - tried the new and old platform.
Arduino - I tried stm32duino, although they added support for the F407Z, but it still doesn't work.

@Dzenik
Copy link

Dzenik commented Jan 25, 2019

I took the settings from the board @xC0000005 and it worked!

@jmz52
Copy link

jmz52 commented Mar 18, 2019

PlatformIO has updated platforms\ststm32 and packages\framework-arduinoststm32.
The arduinoststm32 framework now includes STM32 code v1.5

With a little 'extra_script' magic I was able to compile Marlin using custom 'board' and 'variant' definitions

[env:black_stm32f407ve]
platform = ststm32
framework = arduino
board = blackSTM32F407VET6
extra_scripts = pre:buildroot/share/PlatformIO/scripts/black_stm32f407vet6.py
...

PlatformIO uses 'board' definition from buildroot/share/PlatformIO/boards/blackSTM32F407VET6.json

[platformio]
boards_dir = buildroot/share/PlatformIO/boards

And 'variant'-specific code from buildroot/share/PlatformIO/variants/MARLIN_F407VE
The 'variant'-specific code is copied to packages/framework-arduinoststm32/variants by black_stm32f407vet6.py 'extra-script'

The black_stm32f407vet6.py script

import os,shutil
from SCons.Script import DefaultEnvironment
from platformio import util

env = DefaultEnvironment()
platform = env.PioPlatform()
board = env.BoardConfig()

FRAMEWORK_DIR = platform.get_package_dir("framework-arduinoststm32")
CMSIS_DIR = os.path.join(FRAMEWORK_DIR, "CMSIS", "CMSIS")
assert os.path.isdir(FRAMEWORK_DIR)
assert os.path.isdir(CMSIS_DIR)
assert os.path.isdir("buildroot/share/PlatformIO/variants")

mcu_type = board.get("build.mcu")[:-2]
variant = board.get("build.variant")
series = mcu_type[:7].upper() + "xx"
variant_dir = os.path.join(FRAMEWORK_DIR, "variants", variant)

source_dir = os.path.join("buildroot/share/PlatformIO/variants", variant)
assert os.path.isdir(source_dir)

if not os.path.isdir(variant_dir):
    os.mkdir(variant_dir)
    source_files = os.listdir(source_dir)
    for file_name in source_files:
        full_file_name = os.path.join(source_dir, file_name)
        if os.path.isfile(full_file_name):
            shutil.copy(full_file_name, variant_dir)

Tested on my nice little (100x81mm) shield for black stm32f407ve

@jmz52
Copy link

jmz52 commented Mar 27, 2019

I think I've nailed down the SDIO stability problem.
Sometimes there is a rogue 0xFFFFFFFF reading before actual data.
This results in 4 bytes offset in block's data and lost 4 bytes in SDIO FIFO buffer.
These lost bytes corrupts next reading and so on an so forth.

A patched version for HAL_STM32F1 - jmz52/Marlin@e1a29e3#diff-c12a0ff587f1645efeb7f05563654fbf

@jmz52
Copy link

jmz52 commented Feb 5, 2020

Marlin now supports SDIO on STM32F4 boards (HAL_STM32)
MarlinFirmware/Marlin#16756

@jmz52
Copy link

jmz52 commented May 27, 2020

TFT screen and touch panel support for HAL STM32.
Currently for 320x240 screens but can be extended to 480x320 with appropriate MarlinUI implementation.
Also some extra coding is needed for appropriate FSMC and SPI initialization on STM32F4
MarlinFirmware/Marlin#18130

@xC0000005
Copy link

xC0000005 commented May 27, 2020 via email

@jmz52
Copy link

jmz52 commented May 29, 2020

@xC0000005 you are right, bank parameter depends on chip select pin.
MKS Robin is using NE4, so I've missed the hard-coded settings.

@xC0000005
Copy link

We're getting closer...
IMG_1754

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants