⬆️ Upgrade to the latest everest #78
Comments
|
@the-bay-kay decided to continue tracking the roll forward in #74 since she was going to make the simulator improvements after the roll forward. Since that is issue is super long now, I am using this to track my testing and cleanup (if needed) of the roll forward. ETA tomorrow. |
|
I am first rolling forward the demo script with the patch changes from the previous (June) repo before switching to the new September repo. I will not plan on rolling over the changes related to OCPP (since they will soon be obsolete), but I will plan on getting the payment mode changes in so that they can be reused. While doing that, the
Since this is not running off the charine2e image, it is based on the march release |
|
Actually, the
although when I copy-pasted the offending log to the previous command I did get an error when starting As we can see from the code, this must be because I should expect to see HLC enabled, and I am not. Let's check the config that I'm trying to copy over with the old config... |
|
Argh, no! |
|
Ok, so there were actually a couple of errors:
The challenge here, of course, was that there was no error reported here. Something around "Looking for HLC, not found", would be helpful! After starting the correct script, the SP3 charge session started. |
|
SP1 fails because, although we are using an unencrypted connection between the station and the CSMS, we are still trying to use an encrypted connection between the station and the car, which apparently fails. Does SP1 imply TLS between station and car? It seems like that is a separate flag. If we are in SP1, we don't copy certs around, but that should be OK because the self-signed certs are used by both the car and manager modules in EVerest. So I am still not sure why this fails. |
|
Turning off TLS in the config We can SP1 to work. I believe this is the current state after rolling forward as well. So I am not going to debug this now since there is a workaround, and we can just debug it in the Sept release. Moving on to cleaning up the demo script a bit before rolling forward. |
|
I checked the current demo script on the march release and it is broken for SP1 and SP2 (with ISO 15118-2) because of the cert/TLS issue. I am going to disable them temporarily while cleaning up the demo script, because there is no point in having a broken demo. SP1 and SP2 do work without ISO, but it literally says ISO in the demo name. We should not be promising things we can't deliver. |
shankari
changed the title
Several of the options in the demo do not actually work For example: - Citrine fails to build (even on main) - SP1 (and SP2) won't work with TLS EVerest#78 (comment) Fixes: - Error out before Citrine build with a message indicating that it is not supported - Patch the config for SP1 and SP2 to disable TLS Testing done: - Citrine ``` Cloning citrineos CSMS from https://github.com/citrineos/citrineos-core.git into /var/folders/y5/cx3cfzrd2q116myv9ly86sw1rnlmdj/T/tmp.I8ClXLLd/citrineos-csms and starting it Cloning into 'citrineos-csms'... remote: Enumerating objects: 16467, done. remote: Counting objects: 100% (303/303), done. remote: Compressing objects: 100% (221/221), done. remote: Total 16467 (delta 139), reused 161 (delta 70), pack-reused 16164 (from 1) Receiving objects: 100% (16467/16467), 5.07 MiB | 969.00 KiB/s, done. Resolving deltas: 100% (10346/10346), done. /var/folders/y5/cx3cfzrd2q116myv9ly86sw1rnlmdj/T/tmp.I8ClXLLd/citrineos-csms /var/folders/y5/cx3cfzrd2q116myv9ly86sw1rnlmdj/T/tmp.I8ClXLLd Copying certs into /var/folders/y5/cx3cfzrd2q116myv9ly86sw1rnlmdj/T/tmp.I8ClXLLd/citrineos-csms/Server/data/certificates Validating that the certificates are set up correctly Server/data/certificates/certChain.pem: OK Chain: depth=0: CN=host.docker.internal, O=EVerest, C=DE, DC=CPO (untrusted) depth=1: CN=CPOSubCA2, O=EVerest, C=DE, DC=V2G (untrusted) depth=2: CN=CPOSubCA1, O=EVerest, C=DE, DC=V2G (untrusted) depth=3: CN=V2GRootCA, O=EVerest, C=DE, DC=V2G No patches to apply Build and run CitrineOS does not currently build due to issues with npm dependencies. It is disabled until we roll forward. Apologies for the inconvenience! ``` - Maeve ``` Successfully copied 84.5kB to everest-ac-demo-manager-1:/ext/source/build/dist/share/everest/modules/OCPP201/device_model_storage.db Successfully copied 3.07kB to everest-ac-demo-manager-1:/tmp/ /ext/source /workspace patching file config/config-sil-ocpp201-pnc.yaml ``` And then charging with EIM was successful Signed-off-by: Shankari <[email protected]>
|
Now that we have cleared the decks with #82, we are ready to roll everything forward. I will base my changes and testing on https://github.com/the-bay-kay/everest-demo/tree/rollforward-demo Onward and upward! |
|
While rolling the everest release forward to the one from September, we also need to roll the build-kit image forward. The everest september release was tagged with a commit from Oct 25 There does not appear to be a build-kit release on the same day (this is something we need to bring up). Looking at the options, we see: which is clearly too old more recently, we have All of them seem to have been generated on the same day (oct 30), Note that the build-kit release is after the everest release, if this is an issue, we can roll back to the last stable release, which appears to be |
|
Pulled the release and built it manually to figure out how to change the Dockerfile
Note that trying to run the image directly kept running into issues with the format?!
|
|
The final step is to upload the test script so that we can run it. I double checked the docs, and the instructions are the same as the Same error. Searched to see where the framework is so that we can put it into the I do notice that the framework is used in the pyjosev code and pyjosev does run, so we should be able to figure out where it is running from. But this is not required for the interactive demo, so let's ask a question in the zulip chat and move on. |
|
Tracking the automated test python dependency issue in https://lfenergy.zulipchat.com/#narrow/channel/417679-EVerest.3A-Testing-.26-CI/topic/.60everest.2Eframework.60.20not.20found.20when.20trying.20to.20run.20tests/near/482047129 |
|
For the record, the list of existing This makes me think that maybe the issue is that I have to activate a virtual environment, but it is not clear which one, and why it wasn't activated during build or while following the outlined instructions. |
|
Returning to #78 (comment) and
Per @the-bay-kay and @catarial we should just edit the config directory, and that will be use to generate the device model when the manager is launched. Verifying this by checking that the device model DB does not exist Starting up the manager requires an MQTT server, so we install and run one on the manager This now starts up everything And after killing the entry, we do see the database So we can change the config (if required) in the demo scripts and create the database that way. |
|
Error while building in the CI/CD But the 1.4.2 version is clearly present I wonder if it is the Bingo! Note also that while using this image, we get the error |
|
Ok, so I can confirm that I get the same error while trying to build locally even when I am on an x86 mac Removing the But the SHA of the latest is the same as the SHA for 1.4.2 I guess it pulls the "unknown" image above. Inspecting the image, we get But I am still able to run it on my x86 Ah, it looks like amd64 and x86_64 are compatible (jeez, it has been a really long time since I had to look at processor architectures and instruction sets!) - I now have to use wikipedia and don't know this stuff cold So:
So we can just remove the image architecture. But this doesn't solve our arm problems (at least not yet). |
|
|
Since the architecture name has now changed to `amd64` instead of `x64_64`. This partially reverts EVerest@b0dfa4d Additional details at: EVerest#78 (comment) Testing done: - Before this change `docker compose -f docker-compose.build.yml build` errors out - After this change `docker compose -f docker-compose.build.yml build` does not error out at the same step Signed-off-by: Shankari <[email protected]>
|
Fixed this, but then running out of disk space while building... This is going to be super complicated to fix since we can't really control the everest build or the space available on the github action workers, so I am going to just build locally and push. |
Since the architecture name has now changed to `amd64` instead of `x64_64`. This partially reverts EVerest@b0dfa4d Additional details at: EVerest#78 (comment) Testing done: - Before this change `docker compose -f docker-compose.build.yml build` errors out - After this change `docker compose -f docker-compose.build.yml build` does not error out at the same step Signed-off-by: Shankari <[email protected]>
Since the architecture name has now changed to `amd64` instead of `x64_64`. This partially reverts EVerest@b0dfa4d Additional details at: EVerest#78 (comment) Testing done: - Before this change `docker compose -f docker-compose.build.yml build` errors out - After this change `docker compose -f docker-compose.build.yml build` does not error out at the same step Signed-off-by: Shankari <[email protected]>
Building locally, we get an SSL errorI then tried to run the image and ran into a similar issue - it looks like there are issues with SSL periodically. Failed while building in running container too with an SSL error |
|
Using the committed image, the basic AC charging demo works. And I haven't even tried the OCPP integration. |
Several of the options in the demo do not actually work For example: - Citrine fails to build (even on main) - SP1 (and SP2) won't work with TLS EVerest#78 (comment) Fixes: - Error out before Citrine build with a message indicating that it is not supported - Patch the config for SP1 and SP2 to disable TLS Testing done: - Citrine ``` Cloning citrineos CSMS from https://github.com/citrineos/citrineos-core.git into /var/folders/y5/cx3cfzrd2q116myv9ly86sw1rnlmdj/T/tmp.I8ClXLLd/citrineos-csms and starting it Cloning into 'citrineos-csms'... remote: Enumerating objects: 16467, done. remote: Counting objects: 100% (303/303), done. remote: Compressing objects: 100% (221/221), done. remote: Total 16467 (delta 139), reused 161 (delta 70), pack-reused 16164 (from 1) Receiving objects: 100% (16467/16467), 5.07 MiB | 969.00 KiB/s, done. Resolving deltas: 100% (10346/10346), done. /var/folders/y5/cx3cfzrd2q116myv9ly86sw1rnlmdj/T/tmp.I8ClXLLd/citrineos-csms /var/folders/y5/cx3cfzrd2q116myv9ly86sw1rnlmdj/T/tmp.I8ClXLLd Copying certs into /var/folders/y5/cx3cfzrd2q116myv9ly86sw1rnlmdj/T/tmp.I8ClXLLd/citrineos-csms/Server/data/certificates Validating that the certificates are set up correctly Server/data/certificates/certChain.pem: OK Chain: depth=0: CN=host.docker.internal, O=EVerest, C=DE, DC=CPO (untrusted) depth=1: CN=CPOSubCA2, O=EVerest, C=DE, DC=V2G (untrusted) depth=2: CN=CPOSubCA1, O=EVerest, C=DE, DC=V2G (untrusted) depth=3: CN=V2GRootCA, O=EVerest, C=DE, DC=V2G No patches to apply Build and run CitrineOS does not currently build due to issues with npm dependencies. It is disabled until we roll forward. Apologies for the inconvenience! ``` - Maeve ``` Successfully copied 84.5kB to everest-ac-demo-manager-1:/ext/source/build/dist/share/everest/modules/OCPP201/device_model_storage.db Successfully copied 3.07kB to everest-ac-demo-manager-1:/tmp/ /ext/source /workspace patching file config/config-sil-ocpp201-pnc.yaml ``` And then charging with EIM was successful Signed-off-by: Shankari <[email protected]>
|
There is no option to turn off https in the go module (issue 27332 in https://github.com/golang/go). |
|
build is still too large to fit in Github actions, so I will need to build and push. Even more argument for just saving + load to make progress. |
|
Switched back to the old container store, and I don't see any images. So it must have been building every time. I am not sure how it was building until yesterday and is not building today - unless it is some issue with netskope. But regardless, I cannot build now, and I can't figure out how to get the netskope root CA to install in the container. |
|
Ok so now I have a couple of options:
Let's pull and edit the Dockerfile and then try to build locally. |
|
Trying to use the Trying to use I poke around by launching the base image and then trying to run commands directly in it Just to clarify, the issue is that the alpine image only has a single root CA, which is the one from netskope. The problem is that netskope doesn't seem to support the letsencrypt root CA So So we can access non letencrypt locations or or but anything with letsencrypt fails If we can install the letsencrypt root CA and chain into the image, we should be fine. |
|
ok, so I got the letsencrypt root cert and installed it, and it was apparently already there. |
|
Tried to switch to a debian build instead of alpine. same error I tried both bookworm and bullseye. No luck. |
|
In the argument for not including the
But the automatic redirect to TLS defeats that |
|
Tried to build with docker-compose and push, but the push is running into issues. This seems to be because we don't have a package listed Switching to the build-and-push action at least initially since it looks like it creates it properly. |
|
Switching worked, we now have the new images but they are still not pushing Apparently, this is an intermittent error. Trying again after a bit.... |
|
Ah no it is because we are testing this in US-JOET. Let's temporarily switch to the US-JOET namespace... |
|
And now the push works It would have likely worked with the old version where we used |
|
During cleanup, we could also build from docker-compose and then push using https://github.com/akhilerm/tag-push-action |
* 👷Initial workflow to checkout and build the CSMS This can definitely be polished, but it works now so can unblock EVerest#78 Changes: - Add a new workflow - Configure it with a hardcoded set of images to build for maeve - Apply compile-time patches - Build and push, similar to the original `cicd` workflow ----- full list of changes ----- * Build on pull requests to upgrade branches as well * Handle the two directories correctly Need to explicitly `cd` into `everest-demo` And apply patches from there * Fix repo format * Fix checked out directory format * Again change directory properly before building * Now that build works, re-tag and push * Fix invalid workflow format * Really fix the format * read the env properly (from within the demo) * Login before pushing * Switch to the build-push-action so that push to packages works EVerest#78 (comment) * Actually push the values * Give more permissions to the token To be consistent with https://docs.github.com/en/packages/managing-github-packages-using-github-actions-workflows/publishing-and-installing-a-package-with-github-actions * Change the permission to "admin" So that we can really really push?! * Temporarily switch to US-JOET so we can verify that it works * Split the maeve patches into compile time and runtime And apply only the compile time patch while building the image
* 👷Initial workflow to checkout and build the CSMS This can definitely be polished, but it works now so can unblock EVerest#78 Changes: - Add a new workflow - Configure it with a hardcoded set of images to build for maeve - Apply compile-time patches - Build and push, similar to the original `cicd` workflow ----- full list of changes ----- * Build on pull requests to upgrade branches as well * Handle the two directories correctly Need to explicitly `cd` into `everest-demo` And apply patches from there * Fix repo format * Fix checked out directory format * Again change directory properly before building * Now that build works, re-tag and push * Fix invalid workflow format * Really fix the format * read the env properly (from within the demo) * Login before pushing * Switch to the build-push-action so that push to packages works EVerest#78 (comment) * Actually push the values * Give more permissions to the token To be consistent with https://docs.github.com/en/packages/managing-github-packages-using-github-actions-workflows/publishing-and-installing-a-package-with-github-actions * Change the permission to "admin" So that we can really really push?! * Temporarily switch to US-JOET so we can verify that it works * Split the maeve patches into compile time and runtime And apply only the compile time patch while building the image Signed-off-by: Shankari <[email protected]>
Since it is not working anyway due to lack of disk space EVerest#78 (comment) Signed-off-by: Shankari <[email protected]>
Since it is not working anyway due to lack of disk space EVerest#78 (comment) Signed-off-by: Shankari <[email protected]>
|
At long last, the build werks ❗❗❗ Now to go ahead and run Runs!! But then immediately crashes with Time to start tweaking the config - fortunately, @the-bay-kay has gone through this and figured it out (mostly!) |
|
After changing a bunch of config, EVerest started up and was able to finish a charge session with -1
Now applying those changes and then I will merge the current open PR #83 |
|
Incorporated changes (with copious patches) into the current demo. This is the stock charging profile, without any departure time. Committed as b613ba6 |
|
Renamed the US-JOET images to And changed the tag to Almost done with cleanup and about ready to merge. |
Now that the device model DB is overridden on startup, we cannot configure the CSMS URL by copying over the database any more (b613ba6, EVerest#78 (comment)). Instead we have to edit the `InternalCtrlr.json` file before the server starts up so that it is configured properly in the device model DB that is created at startup. We do this by using `sed` to replace `localhost` with the correct CSMS URL. But the CSMS URL is different based on the the profile (SP1, SP2, SP3) and the CSMS (Maeve vs CitrineOS). So we make the following changes: - remove all the DB override files since they are not relevant any more - instead, define the SP and CSMS specific enviroment variables in the CSMS-specific apply-patches file - we may want to change this to a separate file later if there are other environment variables we want to specify - specifying this in CSMS-specific files means that we can easily support other CSMSes without modifying the demo file - change the demo script to run separate sed commands that modify the `InternalCtrlr.json` with the correct URL for the service profile and CSMS Signed-off-by: Shankari <[email protected]>
This is a partial fix for EVerest#78 There is still much cleanup to be done, but I want to pivot to the MIDAS integration right now Signed-off-by: Shankari <[email protected]>
|
Now that I have merged #83 I am now going to pivot to #79, so I will close this. But for the record, other "nice to have" changes would be to:
|
|
@Abby-Wheelis, you should be able to test the AC, ISO 15118-2 DC and the ISO 15118 + OCPP 201 demos now. |
shankari
changed the title
The text was updated successfully, but these errors were encountered: