Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add menu options for EEPROM size selection #261

Closed
wants to merge 1 commit into from
Closed

Conversation

dbuezas
Copy link
Owner

@dbuezas dbuezas commented Mar 30, 2023

What do you think about this @LaZsolt ?

Relates to #243

@dbuezas
Copy link
Owner Author

dbuezas commented Mar 30, 2023

Board setup for 0 EEPROM if the lib is not included

image

if EEPROM.h is not included, the eeprom emulation registers are never initialised and the board is therefore (I think) setup to emulate zero EEPROM

int lgt_eeprom_size(){
  if ( ECCR & 0x40 ) return  ( 1020 << ( ECCR & 0x3 ) );
  return 0;
}

void setup() {
  Serial.begin(115200);
  Serial.print("lgt_eeprom_size:");
  Serial.println(lgt_eeprom_size());
  delay(1000);
}
void loop() { }
  • Prints: lgt_eeprom_size: 0
  • When setting up EEPROM Emulation to 0 KB, compilation shows: Sketch uses 1940 bytes (6%) of program storage space. Maximum is 31744 bytes. (I came up with this value by adding 1024*2 to the old upload.maximum_size)

If the lib is imported

#define USE_LGT_EEPROM_API
#include <EEPROM.h>

void setup() {
  Serial.begin(115200);
  lgt_eeprom_init(EEPROM_SIZE); // this is the constant set through the menu
  Serial.print("lgt_eeprom_size: ");
  Serial.print(lgt_eeprom_size());
  Serial.println();
}

void loop() { }

Prints lgt_eeprom_size: 8160 (or whatever is set in the menu )

@@ -102,10 +102,6 @@ void lgt8fx8x_init()
// store ivref calibration value (According to the LGT8F328D databook, if VCAL is rewritten, the value of VCAL1 can be lost.)
_VCAL_1_ = VCAL1;

// enable 1KB E2PROM
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@LaZsolt The EEPROM.h library will set this to 1 by default when imported anyway.

If the lib is not imported, the ECCR register will stay at zero, which means "I won't use the EEPROM"

Copy link
Collaborator

@LaZsolt LaZsolt Mar 31, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Incompatible idea. #261 (comment)

@dbuezas
Copy link
Owner Author

dbuezas commented Mar 30, 2023

@dwillmore @jg1uaa @jayzakk, your feedback (and that of anybody else) very welcome :)

Also, I need some help writing a test that:

  • fills the program part of the flash
  • fills EEPROM
  • tests if everything is ok and the EEPROM reads correctly and doesn't overwrite the program

Any willing volunteer? :)

@dbuezas
Copy link
Owner Author

dbuezas commented Mar 30, 2023

Uhm... I just read in our wiki that the bootloader is stored right after the 2 kb reserved for the EEPROM, this whole thing may be cursed...

I don't find any explicit offset set in platforms.txt though

@LaZsolt
Copy link
Collaborator

LaZsolt commented Mar 31, 2023

the bootloader is stored right after the 2 kb reserved for the EEPROM

Not true. The bootloader is stored before the 2kB reserved EEPROM space. So

making the EEPROM space configurable via menu would add another 2k

not true.

The factory burned bootloader started at address 0x7400 and the emulated EEPROM start address is 0x7800 which ending at the end of the FLASH.

Ergo for the different emulated EEPROM size require different bootloader.

All in all if the EEPROM size menu exist it is only could reseve ROM space for such a type of EEPROM emulation software which not using the hardware emulation. In this case the bootloader must start at address 0x7C00 and e.g. 1kB emulater EEPROM start at 0x7400

@LaZsolt
Copy link
Collaborator

LaZsolt commented Mar 31, 2023

Edit:

the bootloader is stored right after the 2 kb reserved for the EEPROM

Not true. The bootloader is stored before the 2kB reserved EEPROM space.

@LaZsolt LaZsolt marked this pull request as ready for review March 31, 2023 07:03
@LaZsolt
Copy link
Collaborator

LaZsolt commented Mar 31, 2023

Somehow I unmarked the draft attribute this pull request. But I don't know how.

@dbuezas dbuezas marked this pull request as draft March 31, 2023 08:47
@dbuezas
Copy link
Owner Author

dbuezas commented Mar 31, 2023

Somehow I unmarked the draft attribute this pull request. But I don't know how.

it's on the top right. Marked as draft again
image

@dbuezas
Copy link
Owner Author

dbuezas commented Apr 1, 2023

Ok, I'm thinking about closing the PR then.

What if one is using an ISP? There's no bootloader in that case so maybe adding a new board type for it would allow using all the space

@SuperUserNameMan
Copy link
Contributor

BTW, I did not follow the impact of the recent updates, but if one of you change the address of the bootloader according to the EEPROM size picked in the menu, please update the https://github.com/dbuezas/lgt8fx/wiki/EEPROM-of-the-LGT8F328p-:-the-important-things-you-need-to-know

@SuperUserNameMan
Copy link
Contributor

SuperUserNameMan commented Apr 2, 2023

Uhm... I just read in our wiki that the bootloader is stored right after the 2 kb reserved for the EEPROM, this whole thing may be cursed...

As mentioned by @LaZsolt , when i wrote the wiki, the bootloader was actually stored BEFORE the EEPROM :

  • bootloader from 0x7400 to 0x77ff (1kB)
  • EEPROM from 0x7800 to 0x7fFF (2kB)

Extending the EEPROM at runtime would overwrite the bootloader, and might also overwrite a part of the program space.

Setting the EEPROM to 0 by default, would leave 2kB of unused blank space AFTER the bootloader, unless you relocate the bootloader.


Here is something I did not think about when i rewrote the EEPROM library : maybe it is possible to swap the bootloader and the EEPROM.

It is possible to modify the EEPROM library to protect the 2 last 1kB pages in which would be relocated the bootloader.
In consequence, we would waste 1kB (since the bootloader is 1kB long), but the size of the EEPROM could be extended without overwriting the bootloader.
Also, setting the EEPROM to 0 by default would leave 3kB of extra space for the program.

edit : and when the EEPROM is enabled, the unused 1kB page just before (or just after) the bootloader could be used as a emulated-fuse space. (as it is possible to write once into each blank cells without triggering destructive page swap)

@dbuezas
Copy link
Owner Author

dbuezas commented Apr 2, 2023

@SuperUserNameMan you are back! :) I sent you Collaborator rights. This allows you to merge PRs and allows ppl in comments to link you even if you are not part of the thread yet.

I did not follow the impact of the recent updates

Nothing related to the bootloader or eeprom except this PR.

maybe it is possible to swap the bootloader and the EEPROM.

It probably is, but this will make this core incompatible with the default bootloader these boards typically ship with. Most likely a bad idea.

Extending the EEPROM at runtime would overwrite the bootloader

I don't understand this: the bootloader is in lower ram than the emulated EEPROM space. Does the EEPROM grow backwards?


I think the only option available would be to make an option to not reserve EEPROM space specifically to be used without the bootloader (i.e while burning via ISP). WDYT?

@SuperUserNameMan
Copy link
Contributor

SuperUserNameMan commented Apr 2, 2023

@SuperUserNameMan you are back! :) I sent you Collaborator rights. This allows you to merge PRs and allows ppl in comments to link you even if you are not part of the thread yet.

My availability might be unreliable for major contribution, though i might still be of use regarding EEPROM questions as long as my memory is not outdated.

maybe it is possible to swap the bootloader and the EEPROM.

It probably is, but this will make this core incompatible with the default bootloader these boards typically ship with. Most likely a bad idea.

Do we have access to the bootloader source-code ?
Isn't the bootloader burned at each sketch upload ?
(edit : indeed these two questions show how rusty i became ...)

Extending the EEPROM at runtime would overwrite the bootloader

I don't understand this: the bootloader is in lower ram than the emulated EEPROM space. Does the EEPROM grow backwards?

Yes, it is extended backwards so that everything remains within the 32kB memory space.

I think the only option available would be to make an option to not reserve EEPROM space specifically to be used without the bootloader (i.e while burning via ISP). WDYT?

Yes, i think you could set the size of the EEPROM to 0 by default, so when the sketch is uploaded using an ISP, the whole 32KB of flash is available for the sketch.

And when uploading a sketch using the bootloader, setting the size of the EEPROM to 0 by default should have no impact other than wasting 2kB of space at the end of the flash. (however, you might want to check if it does not interfere with the initialization code of the EEPROM library)

@SuperUserNameMan
Copy link
Contributor

I don't understand this: the bootloader is in lower ram than the emulated EEPROM space. Does the EEPROM grow backwards?

Do you mean the bootloader is copied into the RAM ?

@LaZsolt
Copy link
Collaborator

LaZsolt commented Apr 2, 2023

The bootloader runs from flash started at address 0x7400. From these address the code can be read with the pgm_read_byte().

But the bootloader code also readabe from RAM address started at 0xB400.

@LaZsolt
Copy link
Collaborator

LaZsolt commented Apr 2, 2023

Crazy idea, but if the EEPROM emulated by software it could be jump up the bootloader code.

Another crazy idea, for that case when the EEPROM size is 0, is that define a fixed adressable read only 1k data block with starting address is the same as the bootloader then the compiler could avoid to put any code here, but the last 2k became useable.

@dbuezas dbuezas closed this Apr 2, 2023
@dbuezas
Copy link
Owner Author

dbuezas commented Apr 2, 2023

Very interesting!
I don't see any offsets in the upload scripts, so it must be the bootloader the one that adds the offset.
If I'm correct, it means that if one uses EEPROM in a sketch uploaded via ISP, the source code would ve overwritten. Right?

@dbuezas dbuezas reopened this Apr 2, 2023
@SuperUserNameMan
Copy link
Contributor

If I'm correct, it means that if one uses EEPROM in a sketch uploaded via ISP, the source code would ve overwritten. Right?

Unless the compiler is aware of the maximum available memory, yes.

@SuperUserNameMan
Copy link
Contributor

SuperUserNameMan commented Apr 2, 2023

Crazy idea, but if the EEPROM emulated by software it could be jump up the bootloader code.

Yes, i was thinking about the same.
I think it is possible to tell to the EEPROM library to avoid the two 1kB pages encompassing the bootloader. (as they work in pair, we must skip 2kB (edit : the EEPROM libray should just skip the before last 1KB page, and the hardware EEPROM module will convert it to a 2KB skip. See next messages bellows )

We then would have EEPROM with theoretical sizes of 1KB, 3KB or 7KB.

number of 1KB pages of the EEPROM theoretical size of the EEPROM actual available bytes and emulated continuous space occupied space into main Flash memory beginning address into Flash memory
0 0 0 0 n/a
1 1024 1020 2048 0x7800
2 n/a n/a n/a 0x7000
3 3072 3060 8192 0x6000
7 7168 7140 16384 0x4000
default 1024 1020 2048 0x7800

Instead of :

number of 1KB pages of the EEPROM theoretical size of the EEPROM actual available bytes and emulated continuous space occupied space into main Flash memory beginning address into Flash memory
0 0 0 0 n/a
1 1024 1020 2048 0x7800
2 2048 2040 4096 0x7000
4 4096 4080 8192 0x6000
8 8192 8160 16384 0x4000
default 1024 1020 2048 0x7800

Another crazy idea, for that case when the EEPROM size is 0, is that define a fixed adressable read only 1k data block with starting address is the same as the bootloader then the compiler could avoid to put any code here, but the last 2k became useable.

If it is possible to tell the compiler to skip the bootloader, that's a good idea !

@SuperUserNameMan
Copy link
Contributor

Oh wait, I think i need to refresh my memory about how the EEPROM is emulated before misleading you on that topic.

@SuperUserNameMan
Copy link
Contributor

SuperUserNameMan commented Apr 2, 2023

Well if you want to skip the bootloader, you just have to hack :
uint16_t lgt_eeprom_continuous_address_to_real_address( uint16_t address )

This function is used to recalculate the EEPROM address so that the library automatically skips the last 4 bytes of each 1KB pages of EEPROM.
It is thus possible to tell it to skip the before last 1KB page of the EEPROM.

Note that the EEPROM library is not aware of the hardware page-swap algorithm which consume 2 Flash pages per 1KB EEPROM page.

Also note that the EEPROM library does NOT think in term of the Flash memory space : it does not care about the actual address into the Flash memory, because it is handled by the hardware EEPROM module which will convert the EEPROM space addresses into Flash space addresses.

So when the EEPROM library will skip the before last 1KB page of its EEPROM space, the hardware module will actually skip 2KB (2 pages) of the Flash memory space (from 0x7000 to 0x7fFF ). Which should preserve the bootloader (and leave 1KB of unused flash memory)

@dbuezas dbuezas closed this Apr 15, 2023
@dbuezas dbuezas deleted the Menu/eeprom_size branch June 14, 2023 07:25
@wollewald
Copy link
Contributor

Hi, I have seen this is closed. I have read it, but still I do not understand if there is a way to get completely rid of the EEPROM in order to use the full 32 KB flash for program code.

@LaZsolt
Copy link
Collaborator

LaZsolt commented Jul 1, 2023

I think if the program uploaded using an ISP then all the 32 KB flash will be available for program code.

@wollewald
Copy link
Contributor

Thanks for the reply.

I tried uploading via LArduinoISP, it does not seem to change anything:

Screenshot 2023-07-01 181307

Screenshot 2023-07-01 181401

I also tried to allocate 0 KB for the EEPROM using:

ECCR = 0x80;
ECCR = 0;

It had no effect. Same with:

lgt_eeprom_init(0);

@LaZsolt
Copy link
Collaborator

LaZsolt commented Jul 1, 2023

The displayed size of uploadable code size comes from here:

328.upload.maximum_size=29696

It can be exceeded but we get a warning.

@wollewald
Copy link
Contributor

I am used from the AVR Arduinos that this max. value is flexible. E.g. if you use the Optiboot bootloader the value is smaller compared to what you get when you don't use it. If it's just a fixed value which uses the default setting of 1 KB EEPROM this explains my confusion. Displaying the real max available flash would be helpful.
Many thanks!!!!

@dbuezas
Copy link
Owner Author

dbuezas commented Jul 1, 2023

So displaying the full 32KB? We can do that if the community agrees

@wollewald
Copy link
Contributor

The main reason for my question is that some people asked if they could use the Arduino Optiboot for the LGT8F328P, because they think it's smaller than the bootloader for the LGT8F328P. Like me, they just at value you get during upload.

@LaZsolt
Copy link
Collaborator

LaZsolt commented Jul 1, 2023

So displaying the full 32KB? We can do that if the community agrees

I think displaying the real available flash will not be easy because uploadig with ISP cannot be detected. I will study the Arduino core solution.

An Optiboot support wish for LGT8F328P: Optiboot/optiboot#359
The current LGT8Fx's bootloader version is made from a very old Optiboot version.
I think if someone could make the short version of the Optiboot for LGT8Fx, then it could be bigger than the ATmega version because of different flash programmig method.

@wollewald
Copy link
Contributor

Just to be sure that I have understood it correctly: it's just the bootloader that takes ~ 3KB and this does not include the 2 KB which are needed for the 1 KB EEPROM?

And thanks for the link!

@LaZsolt
Copy link
Collaborator

LaZsolt commented Jul 1, 2023

The LGT8F328P bootloader less than 1 kB.

@wollewald
Copy link
Contributor

< 1KB - that's great.

If it's difficult to implement a method to display the exact flash which is available, then some documention how the 29696 bytes are calculated and under which conditions this value applies would be helpful. I am by far not so deep in this matter like you, but still I would say I am not an absolute beginner. I simply thought I did something wrong when I changed the size of the EEPROM and the max available flash size didn't change. Therefore, there are probably quite a number of people who have similar problems. I also have not yet finally understood if there is flash reserved for the EEPROM by default, even if I do not include EEPROM.h. From what I read now I would assume that there are roughly 31 KB Flash available if I don't use the EEPROM and 29696 bytes if I use 1 KB of EEPROM. Sorry, to bother you with all these questions, but would really like to understand this.

@dbuezas
Copy link
Owner Author

dbuezas commented Jul 2, 2023

From the wiki article about EEPROM written by @SuperUserNameMan, the flash layout is:

[0, Program..., ...bootloader..., ...EEPROM, 32000]

Trying to use more than 1k of eeprom (taking 2kb of flash) will overwrite the bootloader, because the eeprom "grows backwards".

This means that the only way to have bigger programs is to forego the bootloader (upload via isp).

@wollewald
Copy link
Contributor

Hi, this is helpful. Sorry to be so tenacious, but there is still one thing I am not sure of:

"By default, the size of this emulated EEPROM is set to 1KB, and it can be changed at run-time to 0KB, 1KB,2KB, 4K or 8KB."

Does the default (1KB EEPROM) only apply if I include EEPROM.h? In other words: If I use the bootloader and I don't include EEPROM.h, will no flash be reserved for the EEPROM and therefore I should have ~31KB flash?

@LaZsolt
Copy link
Collaborator

LaZsolt commented Jul 2, 2023

Displaying the real max available flash would be helpful.

I checked the AVR Arduino core.

  • It looks like no possibile way to displaying different available flash size when uploading with ISP or with bootloader.
  • Normally you don't want to overwrite the bootloader when using ISP for upload so the available flash size is the same as in normal case. 29696 bytes.
    (Using ISP for upload there is possibilities to overwrite the bootloader and the emulated EEPROM space. In this case the full flash size available for program, but you get a warning message.)
  • The emulated EEPROM space is always reserved regardless of whether it is used or not. In other words the bootloader won't jump to the end of flash when 0 KB EEPROM is set in runtime.
    (The reserved emulated EEPROM space is at the end of program flash memory. The bootloader precedes the reserved emulated EEPROM space.)

So in my opinion no need to change the displayed maximum upload size.

In summary: The bootloader stored in a fixed location in program flash memory followed by emulated EEPROM space. If EEPROM space changed bigger than 1 KB the bootloader will be overwritten after writing some bytes to the EEPROM spaces. If someone want to use bigger (or smaller) EEPROM space the bootloader must be relocated. This relocation method is not yet invented.

@wollewald
Copy link
Contributor

Very clear - that answers my question perfectly. Thank you!

The other good news: no more questions from side! Thanks for your patience.

@dbuezas
Copy link
Owner Author

dbuezas commented Jul 2, 2023

So to use the full 32kb:

  • Make a program that uses 32Kb
  • Upload via ISP
  • Ignore warnings
  • Don't attempt to write anything to EEPROM

Right?

@LaZsolt
Copy link
Collaborator

LaZsolt commented Jul 2, 2023

Yes.

@dwillmore
Copy link
Collaborator

dwillmore commented Jul 2, 2023 via email

@dbuezas
Copy link
Owner Author

dbuezas commented Jul 2, 2023

No, ATMEL has real EEPROM independent of flash

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants