Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support ESP-IDF Rust Target (for ESP32-C3 especially) #379

Closed
stevefan1999-personal opened this issue Jan 13, 2023 · 9 comments · Fixed by #452 or #460
Closed

Support ESP-IDF Rust Target (for ESP32-C3 especially) #379

stevefan1999-personal opened this issue Jan 13, 2023 · 9 comments · Fixed by #452 or #460

Comments

@stevefan1999-personal
Copy link

Right now it fails to build:

   Compiling socket2 v0.4.7
error[E0432]: unresolved import `libc::SOCK_RAW`
  --> C:\Users\steve\.cargo\registry\src\github.com-1ecc6299db9ec823\socket2-0.4.7\src\sys\unix.rs:64:16
   |
64 | pub(crate) use libc::SOCK_RAW;
   |                ^^^^^^^^^^^^^^ no `SOCK_RAW` in the root

error[E0432]: unresolved import `libc::SOCK_SEQPACKET`
  --> C:\Users\steve\.cargo\registry\src\github.com-1ecc6299db9ec823\socket2-0.4.7\src\sys\unix.rs:66:16
   |
66 | pub(crate) use libc::SOCK_SEQPACKET;
   |                ^^^^^^^^^^^^^^^^^^^^ no `SOCK_SEQPACKET` in the root

error[E0432]: unresolved import `libc::IP_HDRINCL`
  --> C:\Users\steve\.cargo\registry\src\github.com-1ecc6299db9ec823\socket2-0.4.7\src\sys\unix.rs:79:16
   |
79 | pub(crate) use libc::IP_HDRINCL;
   |                ^^^^^^^^^^^^^^^^ no `IP_HDRINCL` in the root

error[E0432]: unresolved import `libc::IP_RECVTOS`
  --> C:\Users\steve\.cargo\registry\src\github.com-1ecc6299db9ec823\socket2-0.4.7\src\sys\unix.rs:89:16
   |
89 | pub(crate) use libc::IP_RECVTOS;
   |                ^^^^^^^^^^^^^^^^ no `IP_RECVTOS` in the root

error[E0432]: unresolved imports `libc::ip_mreq_source`, `libc::IP_ADD_SOURCE_MEMBERSHIP`, `libc::IP_DROP_SOURCE_MEMBERSHIP`
   --> C:\Users\steve\.cargo\registry\src\github.com-1ecc6299db9ec823\socket2-0.4.7\src\sys\unix.rs:117:5
    |
117 |     ip_mreq_source as IpMreqSource, IP_ADD_SOURCE_MEMBERSHIP, IP_DROP_SOURCE_MEMBERSHIP,
    |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  ^^^^^^^^^^^^^^^^^^^^^^^^  ^^^^^^^^^^^^^^^^^^^^^^^^^ no `IP_DROP_SOURCE_MEMBERSHIP` in the root
    |     |                               |
    |     |                               no `IP_ADD_SOURCE_MEMBERSHIP` in the root
    |     no `ip_mreq_source` in the root
    |
help: a similar name exists in the module
    |
117 |     ip_mreq_source as IpMreqSource, IP_ADD_MEMBERSHIP, IP_DROP_SOURCE_MEMBERSHIP,
    |                                     ~~~~~~~~~~~~~~~~~
help: a similar name exists in the module
    |
117 |     ip_mreq_source as IpMreqSource, IP_ADD_SOURCE_MEMBERSHIP, IP_DROP_MEMBERSHIP,
    |                                                               ~~~~~~~~~~~~~~~~~~

error[E0433]: failed to resolve: use of undeclared type `IovLen`
   --> C:\Users\steve\.cargo\registry\src\github.com-1ecc6299db9ec823\socket2-0.4.7\src\sys\unix.rs:790:38
    |
790 |     msg.msg_iovlen = min(bufs.len(), IovLen::MAX as usize) as IovLen;
    |                                      ^^^^^^ use of undeclared type `IovLen`

error[E0433]: failed to resolve: use of undeclared type `IovLen`
   --> C:\Users\steve\.cargo\registry\src\github.com-1ecc6299db9ec823\socket2-0.4.7\src\sys\unix.rs:850:38
    |
850 |     msg.msg_iovlen = min(bufs.len(), IovLen::MAX as usize) as IovLen;
    |                                      ^^^^^^ use of undeclared type `IovLen`

error[E0531]: cannot find unit struct, unit variant or constant `SOCK_RAW` in crate `libc`
   --> C:\Users\steve\.cargo\registry\src\github.com-1ecc6299db9ec823\socket2-0.4.7\src\sys\unix.rs:362:11
    |
362 |     libc::SOCK_RAW,
    |           ^^^^^^^^ not found in `libc`

error[E0531]: cannot find unit struct, unit variant or constant `SOCK_RDM` in crate `libc`
   --> C:\Users\steve\.cargo\registry\src\github.com-1ecc6299db9ec823\socket2-0.4.7\src\sys\unix.rs:364:11
    |
364 |     libc::SOCK_RDM,
    |           ^^^^^^^^ not found in `libc`

error[E0531]: cannot find unit struct, unit variant or constant `SOCK_SEQPACKET` in crate `libc`
   --> C:\Users\steve\.cargo\registry\src\github.com-1ecc6299db9ec823\socket2-0.4.7\src\sys\unix.rs:365:11
    |
365 |     libc::SOCK_SEQPACKET,
    |           ^^^^^^^^^^^^^^ not found in `libc`

error[E0412]: cannot find type `IovLen` in this scope
   --> C:\Users\steve\.cargo\registry\src\github.com-1ecc6299db9ec823\socket2-0.4.7\src\sys\unix.rs:790:63
    |
790 |     msg.msg_iovlen = min(bufs.len(), IovLen::MAX as usize) as IovLen;
    |                                                               ^^^^^^ not found in this scope

error[E0412]: cannot find type `IovLen` in this scope
   --> C:\Users\steve\.cargo\registry\src\github.com-1ecc6299db9ec823\socket2-0.4.7\src\sys\unix.rs:850:63
    |
850 |     msg.msg_iovlen = min(bufs.len(), IovLen::MAX as usize) as IovLen;
    |                                                               ^^^^^^ not found in this scope

error[E0412]: cannot find type `ip_mreqn` in crate `libc`
    --> C:\Users\steve\.cargo\registry\src\github.com-1ecc6299db9ec823\socket2-0.4.7\src\sys\unix.rs:1035:12
     |
1035 |   ) -> libc::ip_mreqn {
     |              ^^^^^^^^ help: a struct with a similar name exists: `ip_mreq`
     |
    ::: C:\Users\steve\.cargo\registry\src\github.com-1ecc6299db9ec823\libc-0.2.135\src\unix\newlib\mod.rs:51:1
     |
51   | / s! {
52   | |     // The order of the `ai_addr` field in this struct is crucial
53   | |     // for converting between the Rust and C types.
54   | |     pub struct addrinfo {
...    |
228  | |     }
229  | | }
     | |_- similarly named struct `ip_mreq` defined here

error[E0422]: cannot find struct, variant or union type `ip_mreqn` in crate `libc`
    --> C:\Users\steve\.cargo\registry\src\github.com-1ecc6299db9ec823\socket2-0.4.7\src\sys\unix.rs:1037:75
     |
1037 |           crate::socket::InterfaceIndexOrAddress::Index(interface) => libc::ip_mreqn {
     |                                                                             ^^^^^^^^ help: a struct with a similar name exists: `ip_mreq`
     |
    ::: C:\Users\steve\.cargo\registry\src\github.com-1ecc6299db9ec823\libc-0.2.135\src\unix\newlib\mod.rs:51:1
     |
51   | / s! {
52   | |     // The order of the `ai_addr` field in this struct is crucial
53   | |     // for converting between the Rust and C types.
54   | |     pub struct addrinfo {
...    |
228  | |     }
229  | | }
     | |_- similarly named struct `ip_mreq` defined here

error[E0422]: cannot find struct, variant or union type `ip_mreqn` in crate `libc`
    --> C:\Users\steve\.cargo\registry\src\github.com-1ecc6299db9ec823\socket2-0.4.7\src\sys\unix.rs:1042:77
     |
1042 |           crate::socket::InterfaceIndexOrAddress::Address(interface) => libc::ip_mreqn {
     |                                                                               ^^^^^^^^ help: a struct with a similar name exists: `ip_mreq`
     |
    ::: C:\Users\steve\.cargo\registry\src\github.com-1ecc6299db9ec823\libc-0.2.135\src\unix\newlib\mod.rs:51:1
     |
51   | / s! {
52   | |     // The order of the `ai_addr` field in this struct is crucial
53   | |     // for converting between the Rust and C types.
54   | |     pub struct addrinfo {
...    |
228  | |     }
229  | | }
     | |_- similarly named struct `ip_mreq` defined here

error[E0063]: missing field `sin_len` in initializer of `sockaddr_in`
   --> C:\Users\steve\.cargo\registry\src\github.com-1ecc6299db9ec823\socket2-0.4.7\src\sockaddr.rs:225:27
    |
225 |         let sockaddr_in = sockaddr_in {
    |                           ^^^^^^^^^^^ missing `sin_len`

error[E0063]: missing field `sin6_len` in initializer of `sockaddr_in6`
   --> C:\Users\steve\.cargo\registry\src\github.com-1ecc6299db9ec823\socket2-0.4.7\src\sockaddr.rs:260:28
    |
260 |         let sockaddr_in6 = sockaddr_in6 {
    |                            ^^^^^^^^^^^^ missing `sin6_len`

Some errors have detailed explanations: E0063, E0412, E0422, E0432, E0433, E0531.
For more information about an error, try `rustc --explain E0063`.
error: could not compile `socket2` due to 17 previous errors
@Thomasdezeeuw
Copy link
Collaborator

Does ESP32 have an OS we can target?

@stevefan1999-personal
Copy link
Author

@Thomasdezeeuw Uh, yes and no. ESP32 is a family of single board computers which is of course bare metal and no_std, but the ESP-IDF development kit, does feature FreeRTOS on with couple more such as a Virtual filesystem component - ESP32 - — ESP-IDF Programming Guide latest documentation (espressif.com), and some people hacked together to introduce standard library support around the ESP-IDF ecosystem, see STD support for the ESP-IDF framework by ivmarkov · Pull Request #87666 · rust-lang/rust (github.com). Notice ESP-IDF is best coupled with ESP32-C3 right now, which is easily obtainable from every corner of the world at any part of the world for around $7.

While the arcanish ESP-IDF target does mean that some level of OS support (in the sense of a RTOS, however), meaning things like std::io, std::fs does work to some degree, but not everything is coming out of the box as a wholesale OS, as for example, I don't think ESP-IDF enabled IPv6 with LwIP out of the box, nor does std::os::env works. Therefore, the situation it's quite complicated to say the least.

To me specifically I'm experimenting smol-rs/smol: A small and fast async runtime for Rust (github.com) on ESP32-C3 and I was successful to some degree, yet here this specific socket2 library is blocking the compilation of async-io, rendering some of the libraries such as async-net which has a dependency of async-io and thus transitive dependency on socket2, so smol is impossible to use as a whole right now.

@stevefan1999-personal stevefan1999-personal changed the title Support ESP32 Rust Targets Support ESP-IDF Rust Target (for ESP32-C3 especially) Jan 14, 2023
@Thomasdezeeuw
Copy link
Collaborator

@Thomasdezeeuw Uh, yes and no. ESP32 is a family of single board computers which is of course bare metal and no_std, but the ESP-IDF development kit, does feature FreeRTOS on with couple more such as a Virtual filesystem component - ESP32 - — ESP-IDF Programming Guide latest documentation (espressif.com), and some people hacked together to introduce standard library support around the ESP-IDF ecosystem, see STD support for the ESP-IDF framework by ivmarkov · Pull Request #87666 · rust-lang/rust (github.com). Notice ESP-IDF is best coupled with ESP32-C3 right now, which is easily obtainable from every corner of the world at any part of the world for around $7.

Socket2 is wrapper around the OS's network stack, i.e. it just wraps socket(2) and related system calls. It doesn't implemented a networking stack itself.

While the arcanish ESP-IDF target does mean that some level of OS support (in the sense of a RTOS, however), meaning things like std::io, std::fs does work to some degree, but not everything is coming out of the box as a wholesale OS, as for example, I don't think ESP-IDF enabled IPv6 with LwIP out of the box, nor does std::os::env works. Therefore, the situation it's quite complicated to say the least.

We don't need too much from std lib, some types from std::net, but those are OS independent.

To me specifically I'm experimenting smol-rs/smol: A small and fast async runtime for Rust (github.com) on ESP32-C3 and I was successful to some degree, yet here this specific socket2 library is blocking the compilation of async-io, rendering some of the libraries such as async-net which has a dependency of async-io and thus transitive dependency on socket2, so smol is impossible to use as a whole right now.

I'm not too familiar with smol or async-net/io my self so I can't say much about it. But perhaps it would be easier to simply not compile the networking side on ESP32 targets? Unless you actually need networking.

If you do need networking then we need good libc support and wrappers for various system calls (usually provided by libc). I would start by removing compilation of everything that isn't available at the moment and start slowing adding things that are available.

@Thomasdezeeuw
Copy link
Collaborator

Since this target doesn't (seem to) have an OS we won't support it in Socket2.

@nevi-me
Copy link

nevi-me commented Feb 25, 2023

This seems relevant: afeb723. It's a patch to make socket2 work on espidf.
I came across it while also struggling with the error, and looking around to see if there's any solution.

Apologies for tagging you, @ivmarkov is this something upstreamable? Or is it better to keep it in the esp-rs-compat org?

For more context, I'm having trouble compiling https://github.com/project-chip/matter-rs for an esp32-c3 device.

@Thomasdezeeuw
Copy link
Collaborator

This seems relevant: afeb723. It's a patch to make socket2 work on espidf. I came across it while also struggling with the error, and looking around to see if there's any solution.

Apologies for tagging you, @ivmarkov is this something upstreamable? Or is it better to keep it in the esp-rs-compat org?

From my point of view it's Ok to upstream, ideally with a CI (at least with cargo check), but without it it's also fine. The downside of having no CI though is that any new release of Socket2 might break compilation on the target.

For more context, I'm having trouble compiling https://github.com/project-chip/matter-rs for an esp32-c3 device.

@ivmarkov
Copy link

I was always planning to upstream that. The one thing holding me off from doing it during the last couple of months was that in the meantime, the mainline of socket2 got updated to V0.5, while I still need the 0.4.X branch patched as well, because async-io still uses V0.4.X.

@Thomasdezeeuw - would you accept two PRs then? One for the v0.4.x branch (the one that I would use), and then another - for the master branch? You know - so that we are prepared once async-io upgrades its socket2 dependency?

As for running cargo check with target_os = "espidf" - I would be more than happy to touch socket2's CI for that (the riscv32imc-esp-espidf is an upstreamed Tier 3 target in Rust STD since > 1 year now so that would work BUT would require Cargo nightly and its -Zbuild-std) - yet - I can live without cargo check either. I mean, Rust STD is not even built for riscv32imc-esp-espidf as it is in Tier 3. Neither is libc, for that target.

@Thomasdezeeuw
Copy link
Collaborator

I was always planning to upstream that. The one thing holding me off from doing it during the last couple of months was that in the meantime, the mainline of socket2 got updated to V0.5, while I still need the 0.4.X branch patched as well, because async-io still uses V0.4.X.

@Thomasdezeeuw - would you accept two PRs then? One for the v0.4.x branch (the one that I would use), and then another - for the master branch? You know - so that we are prepared once async-io upgrades its socket2 dependency?

Yes, but please send it to target v0.5.x first and then we can backport that. Btw I'm going to try and release v0.5 this week(end), so at that point async-io can also upgrade (if it wants to).

As for running cargo check with target_os = "espidf" - I would be more than happy to touch socket2's CI for that (the riscv32imc-esp-espidf is an upstreamed Tier 3 target in Rust STD since > 1 year now so that would work BUT would require Cargo nightly and its -Zbuild-std) - yet - I can live without cargo check either. I mean, Rust STD is not even built for riscv32imc-esp-espidf as it is in Tier 3. Neither is libc, for that target.

Running with nightly is fine. You probably want something we use to have for Redox: 14be1bc (the part that's deleted in that commit).

@jasta
Copy link
Contributor

jasta commented Jul 14, 2023

Hi @Thomasdezeeuw, me again :) I've got a patch ready for the main branch and also the backported patch again v0.4.9 (which is needed because tokio latest appears to still target socket2 0.4.9). I'll post a PR soon once I confirm the master one actually works.

jasta added a commit to jasta/socket2 that referenced this issue Jul 14, 2023
Smoke tested on esp32c3 dev board.  I've also tested a similar patch
backported to v0.4.9 with much greater functionality including tokio +
mio with other patches I've been working on and it's fully working.

Closes rust-lang#379
Thomasdezeeuw pushed a commit that referenced this issue Jul 15, 2023
Smoke tested on esp32c3 dev board.  I've also tested a similar patch
backported to v0.4.9 with much greater functionality including tokio +
mio with other patches I've been working on and it's fully working.

Closes #379
jasta added a commit to jasta/socket2 that referenced this issue Jul 31, 2023
Not sure how this got missed in rust-lang#452, it was definitely compiling
before.

Properly closes rust-lang#379.
jasta added a commit to jasta/socket2 that referenced this issue Jul 31, 2023
This was missed in rust-lang#452 because I wasn't testing with feature="all"
enabled for my small socket2 test.  For the full tokio integration I was
using v0.4.x which didn't need this fix.

Properly closes rust-lang#379.
Thomasdezeeuw pushed a commit that referenced this issue Jul 31, 2023
This was missed in #452 because I wasn't testing with feature="all"
enabled for my small socket2 test.  For the full tokio integration I was
using v0.4.x which didn't need this fix.

Properly closes #379.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
5 participants