Add Zephyr Support #19003

elcritch · 2021-10-16T02:44:01Z

Porting Nim to run on Zephyr, including networking. Follow up on issue 18684.

Squashed commit of the following:
tweaking more memory / malloc things
revert back bitmasks
tweaking nim to use kernel heap as C malloc doesn't work
fixing socket polling on zephyr
cleanup getting maximum sockets for process or for rtos'es
reorganizing and fixing net for async / system
merge netlite changes back into nativesockets
merge netlite changes back into nativesockets
reverting native sockets back
tweaking nim / zephyr network
adding option to run 'net-lite' from linux
bridging zephyr's max connections
fixing net errors
fixing compilation with getAddrString
fixing compilation with getAddrString
experimenting with a nativesockets_lite ... getAddrString
experimenting with a nativesockets_lite ... getAddrString
experimenting with a nativesockets_lite ... getLocalAddr
experimenting with a nativesockets_lite ... getLocalAddr
experimenting with a nativesockets_lite ...
add note regarding incorrect FreeRTOS Sockadd_in fields
changing to NIM_STATIC_ASSERT
cleaning up the static_assert error messages
cleaning up the static_assert error messages
setting up static assert ftw!
testing compile time asserts
reworking Sockaddr objects to more closely match various platforms
reworking Sockaddr objects to more closely match various platforms
reworking Sockaddr objects to more closely match various platforms
finding missing items (issue #18684)
fixup posix constants (issue #18684)
adding plumbing for zephyr os (issue #18684)
adding plumbing for zephyr os (issue #18684)

Includes changes to `std/net`. Squashed commit of the following: tweaking more memory / malloc things revert back bitmasks tweaking nim to use kernel heap as C malloc doesn't work fixing socket polling on zephyr cleanup getting maximum sockets for process or for rtos'es reorganizing and fixing net for async / system merge netlite changes back into nativesockets merge netlite changes back into nativesockets reverting native sockets back tweaking nim / zephyr network adding option to run 'net-lite' from linux bridging zephyr's max connections fixing net errors fixing compilation with getAddrString fixing compilation with getAddrString experimenting with a nativesockets_lite ... getAddrString experimenting with a nativesockets_lite ... getAddrString experimenting with a nativesockets_lite ... getLocalAddr experimenting with a nativesockets_lite ... getLocalAddr experimenting with a nativesockets_lite ... add note regarding incorrect FreeRTOS Sockadd_in fields changing to NIM_STATIC_ASSERT cleaning up the static_assert error messages cleaning up the static_assert error messages setting up static assert ftw! testing compile time asserts reworking Sockaddr objects to more closely match various platforms reworking Sockaddr objects to more closely match various platforms reworking Sockaddr objects to more closely match various platforms finding missing items (issue nim-lang#18684) fixup posix constants (issue nim-lang#18684) adding plumbing for zephyr os (issue nim-lang#18684) adding plumbing for zephyr os (issue nim-lang#18684)

elcritch · 2021-10-16T02:48:10Z

lib/posix/posix.nim

  importc:"(char *)$1", header: "<arpa/inet.h>".}
-proc inet_pton*(a1: cint, a2: cstring, a3: pointer): cint {.
+proc inet_pton*(a1: TSa_Family, a2: cstring, a3: ptr InAddr | ptr In6Addr): cint {.


This isn't directly related to Zephyr, however I found it very confusing to know when to pass ptr InAddr or TSa_Family.

This is a breaking change and unrelated, please undo. Feel free to create another PR that adds an overload where the types have been improved.

Araq · 2021-10-16T15:22:46Z

lib/pure/asyncdispatch.nim

@@ -1972,16 +1972,3 @@ proc activeDescriptors*(): int {.inline.} =

 when defined(posix):
  import posix
-
-when defined(linux) or defined(windows) or defined(macosx) or defined(bsd):


Please keep this code.

This is one last bit I'm still struggling with. Previously this code lived elsewhere. However, the same value is used in a couple of places but getrlimit is not available in Zephyr/FreeRTOS. I'd moved this bit of code to nativesockets.nim to reuse it in ioselectors_X.nim. However that's causing a build failure in some of the tests.

Araq · 2021-10-16T15:25:33Z

lib/system/bitmasks.nim

+    # when defined(useMalloc):
+      # when defined(amd64): 16 
+      # else: 8
+    # else: 16


I think this code was better before.

Ooops, this looks like a mistake. It should be reverted.

Actually it might be what was breaking the "thread local storage" during tests.

Araq · 2021-10-16T15:26:56Z

It fails with

too large thread local storage size requested,
use -d:"nimTlsSize=X" to setup even more or stop using unittest.nim

for mysterious reasons, probably because alignment can really not just be "4" for every target.

Includes changes to `std/net`. Squashed commit of the following: tweaking more memory / malloc things revert back bitmasks tweaking nim to use kernel heap as C malloc doesn't work fixing socket polling on zephyr cleanup getting maximum sockets for process or for rtos'es reorganizing and fixing net for async / system merge netlite changes back into nativesockets merge netlite changes back into nativesockets reverting native sockets back tweaking nim / zephyr network adding option to run 'net-lite' from linux bridging zephyr's max connections fixing net errors fixing compilation with getAddrString fixing compilation with getAddrString experimenting with a nativesockets_lite ... getAddrString experimenting with a nativesockets_lite ... getAddrString experimenting with a nativesockets_lite ... getLocalAddr experimenting with a nativesockets_lite ... getLocalAddr experimenting with a nativesockets_lite ... add note regarding incorrect FreeRTOS Sockadd_in fields changing to NIM_STATIC_ASSERT cleaning up the static_assert error messages cleaning up the static_assert error messages setting up static assert ftw! testing compile time asserts reworking Sockaddr objects to more closely match various platforms reworking Sockaddr objects to more closely match various platforms reworking Sockaddr objects to more closely match various platforms finding missing items (issue nim-lang#18684) fixup posix constants (issue nim-lang#18684) adding plumbing for zephyr os (issue nim-lang#18684) adding plumbing for zephyr os (issue nim-lang#18684)

…yr-os

Includes changes to `std/net`. Squashed commit of the following: tweaking more memory / malloc things revert back bitmasks tweaking nim to use kernel heap as C malloc doesn't work fixing socket polling on zephyr cleanup getting maximum sockets for process or for rtos'es reorganizing and fixing net for async / system merge netlite changes back into nativesockets merge netlite changes back into nativesockets reverting native sockets back tweaking nim / zephyr network adding option to run 'net-lite' from linux bridging zephyr's max connections fixing net errors fixing compilation with getAddrString fixing compilation with getAddrString experimenting with a nativesockets_lite ... getAddrString experimenting with a nativesockets_lite ... getAddrString experimenting with a nativesockets_lite ... getLocalAddr experimenting with a nativesockets_lite ... getLocalAddr experimenting with a nativesockets_lite ... add note regarding incorrect FreeRTOS Sockadd_in fields changing to NIM_STATIC_ASSERT cleaning up the static_assert error messages cleaning up the static_assert error messages setting up static assert ftw! testing compile time asserts reworking Sockaddr objects to more closely match various platforms reworking Sockaddr objects to more closely match various platforms reworking Sockaddr objects to more closely match various platforms finding missing items (issue nim-lang#18684) fixup posix constants (issue nim-lang#18684) adding plumbing for zephyr os (issue nim-lang#18684) adding plumbing for zephyr os (issue nim-lang#18684)

…yr-os

elcritch · 2021-10-21T06:44:19Z

@Araq the checks are passing now and I cleaned up the issues in the previous comments. Let me know how it looks now.

lib/system/mm/malloc.nim

lib/pure/nativesockets.nim

Araq · 2021-10-22T09:14:02Z

lib/posix/posix.nim

+when defined(linux) or defined(windows) or defined(macosx) or defined(bsd):
+  template maxDescriptors*(): int =
+    ## Returns the maximum number of active file descriptors for the current
+    ## process. This involves a system call. For now `maxDescriptors` is
+    ## supported on the following OSes: Windows, Linux, OSX, BSD.
+    var fdLim: RLimit
+    var res = int(getrlimit(RLIMIT_NOFILE, fdLim))
+    if res >= 0:
+      res = int(fdLim.rlim_cur) - 1
+    res
+when defined(zephyr) or defined(freertos):
+  template maxDescriptors*(): int =
+    FD_MAX
+


This should be moved to somewhere else as maxDescriptors is not part of POSIX.

Done. Move it to selectors.nim, it's the only place it's used.

Araq · 2021-10-22T09:17:28Z

lib/system/ansi_c.nim

+    if result == nil:
+      c_free(p)


Hmm, looks like the standard says:

If there is not enough memory, the old memory block is not freed and null pointer is returned.

Looks like i should remove the free on failure.

Removed the extra free.

elcritch · 2021-10-23T21:25:54Z

@Araq ok got those fixes in and it's all compiling again.

@konsumlamm

Concerns these changes: nim-lang/Nim@727c637...340b5a1 Excluded changes are: nim-lang/Nim#18963 nim-lang/Nim#19003 nim-lang/Nim#19043 nim-lang/Nim#19055 nim-lang/Nim#19053 nim-lang/Nim#19064 nim-lang/Nim#18642 nim-lang/Nim#19062 nim-lang/Nim#19082 nim-lang/Nim#19090 nim-lang/Nim#19077 nim-lang/Nim#19021 nim-lang/Nim#19100 nim-lang/Nim#19102 nim-lang/Nim#19111 nim-lang/Nim#19115 nim-lang/Nim#19133 nim-lang/Nim#19142 nim-lang/Nim#19158 nim-lang/Nim#19129 nim-lang/Nim#19137 nim-lang/Nim#19168 nim-lang/Nim#19156 nim-lang/Nim#19147 nim-lang/Nim#19180 nim-lang/Nim#19183 nim-lang/Nim#19182 nim-lang/Nim#19187 nim-lang/Nim#19179 nim-lang/Nim#19209 nim-lang/Nim#19210 nim-lang/Nim#19207 nim-lang/Nim#19219 nim-lang/Nim#19195 nim-lang/Nim#19212 nim-lang/Nim#19134 nim-lang/Nim#19235 nim-lang/Nim#19252 nim-lang/Nim#19196 nim-lang/Nim#19295 nim-lang/Nim#19301 nim-lang/Nim#19181 nim-lang/Nim#17223 nim-lang/Nim#19370 nim-lang/Nim#19385 nim-lang/Nim#19307 nim-lang/Nim#19394 nim-lang/Nim#19399 nim-lang/Nim#19390 nim-lang/Nim#19407 nim-lang/Nim#19419 nim-lang/Nim#19421 nim-lang/Nim#19363 nim-lang/Nim#19406 nim-lang/Nim#19431 nim-lang/Nim#19455 nim-lang/Nim#19461 nim-lang/Nim@cb894c7 nim-lang/Nim#19462 nim-lang/Nim#19442 nim-lang/Nim#19437 nim-lang/Nim#19433 nim-lang/Nim#19512 nim-lang/Nim#19487 nim-lang/Nim#19543 Excluded changes include major changes which require more consideration and changes which don't apply to the current code anymore but could be worth porting over still. Excluded changes which only change the identifier casing in tests or only concern code removed in nimskull aren't listed. Begin commit listing: use two underscores for easy demangling [backport:1.6] (#19028) Add Elbrus 2000 architecture (#19024) * Add Elbrus 2000 architecture * Add e2k to niminst * Update compiler/installer.ini Co-authored-by: Andreas Rumpf <[email protected]> remove exception (#18906) allow converting static vars to `openArray` (#19035) When assigning constant output to a seq, and then passing that static seq to other functions that take `openArray`, the compiler may end up producing errors, as it does not know how to convert `static[seq[T]]` to `openArray[T]`. By ignoring the `static` wrapper on the type for the purpose of determining data memory location and length, this gets resolved cleanly. Unfortunately, it is relatively tricky to come up with a minimal example, as there are followup problems from the failing conversion, e.g., this may lead to `internal error: inconsistent environment type`, instead of the relevant `openArrayLoc` error message. use the correct header for TIOCGWINSZ on Solaris (#19037) Minor update to terminal docs (#19056) * Update terminal.nim - Added some extra docs to cursorUp/Down/Forward/Backward - I was able to use hideCursor and showCursor without adding stdout, removed the parameter - Added docs to terminalHeight()* and terminalWidth()* * Update lib/pure/terminal.nim Co-authored-by: konsumlamm <[email protected]> * Update lib/pure/terminal.nim Co-authored-by: konsumlamm <[email protected]> * Added back f: file to cursor movement * Removed unnecessary comments Co-authored-by: konsumlamm <[email protected]> fix a tiny formating issue in doc/destructors.rst (#19058) fix a tiny code snippet formatting issue in `doc/constructors.rst`, again (#19065) Fix nimIdentNormalize, fixes #19067 (#19068) * Make nimIdentNormalize return "" when passed ""; fixes #19067 Fixes #19067 * Add tests for nimIdentNormalize fix #18971 (#19070) [backport:1.6] since the example code return value from global variable, instead of first argument, the `n.len` is 1 which causes compiler crashes. fixes #19000 (#19032) * fixes #19000 * progress fix #18410 (Errors initializing an object of RootObj with the C++ backend) [backport] (#18836) * fix #18410 * one line comment * typo * typo * cover cpp update numbers of lifetime-tracking hooks in doc/destructors.rst (#19088) bootstrapping Nim compiler with `cpp --gc:orc` (#19087) libs/impore/re: Add note about the requirement of `matches` to be pre-allocated (#19081) Add few runnableExamples for `findBounds` for clarity. Fixes nim-lang/Nim#18775 Add test for issue 15435 (#19079) * Add test for issue 15435 Closes nim-lang/Nim#15435. * Specify bug # in comment Addresses nim-lang/Nim#19079 (comment) manual: Document the use of `static` as a proc call (#19084) * manual: Document the use of `static` as a proc call Also adds tests. Fixes nim-lang/Nim#16987 . * Update doc/manual.rst Co-authored-by: konsumlamm <[email protected]> * Use the "bug #NNNN" comment syntax for consistency Ref: https://nim-lang.github.io/Nim/contributing.html#writing-tests-stdlib > Always refer to a GitHub issue using the following exact syntax: bug for tooling. * manual: Undocument usage of foo.static foo.static and foo.static() are not expected to work. Ref: https://github.com/nim-lang/Nim/pull/19084/files#r741203578 Co-authored-by: konsumlamm <[email protected]> manual: Document that comma propagates the default values of parameters (#19080) * manual: Document that comma propagates the default values of parameters Fixes nim-lang/Nim#15949. * Use the "bug #NNNN" comment syntax for consistency Ref: https://nim-lang.github.io/Nim/contributing.html#writing-tests-stdlib > Always refer to a GitHub issue using the following exact syntax: bug for tooling. fixes #19011 [backport:1.6] (#19114) Add deprecation pragmas in lib/deprecated/pure (#19113) Deprecate `std/sharedlist` and `std/sharedtables` (#19112) fix nimindexterm in rst2tex/doc2tex [backport] (#19106) * fix nimindexterm (rst2tex/doc2tex) [backport] * Add support for indexing in rst Call {.cursor.} a pragma. (#19116) * Call {.cursor.} a pragma. Its hard to find .curser annotation while googling because all other things like it are called pragmas. See https://nim-lang.org/docs/manual.html#pragmas Also the . in front of the name makes it hard to find and search for. Can we just call it cursor pragma? * Small fix for comment. Remove tlsEmulation enabled from Windows + GCC config (#19119) [backport:1.6] This flag has a very significant performance impact on programs compiled with --threads:on. It is also apparently not needed anymore for standard circumstances. Can we remove the config? See nim-lang/Nim#18146 (comment) for discussion and perf impact. [backport:1.6] Add security tip for setCookie (#19117) * Add security tip for setCookie * Update lib/pure/cookies.nim Co-authored-by: Dominik Picheta <[email protected]> * Update lib/pure/cookies.nim Co-authored-by: konsumlamm <[email protected]> Co-authored-by: Andreas Rumpf <[email protected]> Co-authored-by: Dominik Picheta <[email protected]> Co-authored-by: konsumlamm <[email protected]> correct cookie docs (#19122) refactoring: orc can use getThreadId() (#19123) * refactoring: orc can use getThreadId() * progress fixed colorNames sorting mistake (#19125) [backport] update manual (#19130) [backport] Merge file size fields correctly on Windows (#19141) * Merge file size fields correctly on Windows Merge file size fields correctly on Windows - Merge the two 32-bit file size fields from `BY_HANDLE_FILE_INFORMATION` correctly in `rawToFormalFileInfo`. - Fixes #19135 * Update os.nim Fix punycode.decode function (#19136) * Refactor: rename proc to func * Fix punycode.decode function This function could only properly decode punycodes containing a single encoded unicode character. As soon as there was more than one punycode character group to decode it produced invalid output - the number of characters was correct, but their position was not. * Update tpunycode.nim Co-authored-by: Clay Sweetser <[email protected]> Fix undeclared 'SYS_getrandom' on emscripten (#19144) wrong spaces (3 => 2) (#19145) `caseStmtMacros` no longer experimental, experimental manual refactor (#19173) * `caseStmtMacros` no longer experimental, experimental manual refactor * Update doc/manual.rst * apply review suggestions * apply review Co-authored-by: Andreas Rumpf <[email protected]> fix inline syntax highlighting in system.nim (#19184) swap port to correct port order (#19177) Co-authored-by: Jaremy Creechley <[email protected]> feat: TLS-ALPN wrappers for OpenSSL (#19202) Co-authored-by: Iced Quinn <[email protected]> misc bugfixes [backport:1.2] (#19203) treat do with pragmas but no parens as proc (#19191) fixes #19188 [format minor] remove unnecessary spaces (#19216) Making TCC work again on Windows --cpu:amd64 - fix #16326 (#19221) * fix #16326 * removing comments fixes a converter handling regression that caused private converters to leak into client modules; fixes #19213; [backport:1.6] (#19229) Add support for LoongArch (#19223) * Add support for LoongArch * Update compiler/installer.ini Co-authored-by: Andreas Rumpf <[email protected]> remove `std/sharedstrings` (#19228) * remove std/sharedstrings it has been broken since 0.18.0 * rephrase the changelog entry add comments to spawn and pinnedSpawn (#19230) `spawn` uses `nimSpawn3` internally and `pinnedSpawn` uses `nimSpawn4` internally. I comment it in order to help contributors get the gist of its functionality. fixes an old ARC bug: the produced copy/sink operations don't copy the hidden type field for objects with enabled inheritance; fixes #19205 [backport:1.6] (#19232) nimRawSetjmp: support Windows (#19197) * nimRawSetjmp: support Windows Using `_setjmp()` directly is required to avoid some rare (but very annoying) exception-related stack corruption leading to segfaults on Windows, with Mingw-w64 and SEH. More details: status-im/nimbus-eth2#3121 Also add "nimBuiltinSetjmp" - mostly for benchmarking. * fix for Apple's Clang++ Revert "swap port to correct port order (#19177)" (#19234) This reverts commit 0d0c249. move toDeque to after addLast (#19233) [backport:1.0] Changes the order of procs definitions in order to avoid calling an undefined proc. let Nim support Nimble 0.14 with lock-file support [backport:1.6] (#19236) nimc.rst: fix table markup (#19239) Various std net improvements (#19132) * Variant of that works with raw IpAddresses. - Add doc tests for new net proc's. - Aadd recvFrom impl - Add recvFrom impl -- tweak handling data var - Update lib/pure/net.nim Co-authored-by: Dominik Picheta <[email protected]> - cleaning up sendTo args - remove extra connect test - cleaning up sendTo args - fix inet_ntop test - fix test failing - byte len * fix test failing - byte len * debugging odd windows build failure * debugging odd windows build failure * more experiments to figure out the windows failure * try manual assigment on InAddr Co-authored-by: Jaremy Creechley <[email protected]> fix bug #14468 zero-width split (#19248) basicopt.txt: Unify the format (#19251) fix: fixes bug in CVerifyPeerUseEnvVars (#19247) Previously CVerifyPeerUseEnvVars was not being passed into scanSslCertificates, which meant that we weren't scanning additional certificate locations given via the SSL_CERT_FILE and SSL_CERT_DIR environment variables suggestion to respect typedarray type (#19257) * suggestion to respect typedarray * Update jssys.nim Co-authored-by: Sven Keller <[email protected]> fix #19244 - solves the problem of the InAddr object constructor in Windows. (#19259) * Update winlean.nim * Update tnet_ll.nim Fixed typo in manual.rst unsafeAssign->uncheckedAssign. Fixes part 1 of #19266 (#19267) use uppercase "type" for Proxy-Authorization header (#19273) Some servers will reject authorization requests with a lowercase "basic" type. Changing to "Basic" seems to solve these issues. https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Proxy-Authorization Update colors.nim (#19274) * Update colors.nim Added `lightgray` alias to `lightgrey` and `...grey`aliases for the rest of the gray colors. Added color `rebeccapurple`. Fixed the incorrect values for the `PaleVioletRed` and `MediumPurple` colors. This module should now be matching the CSS colors. I used the seq[tuple] syntax for defining the names. * Document colors changes. Extract runnables that specify `doccmd` (#19275) [backport:1.6] Fix build on FreeBSD/powerpc (#19282) It's currently misdetected as powerpc64. Fix #19107 (#19286) [backport] fixes grammar typos [backport] (#19289) fix 19292 (#19293) Fix #19297 - fixing broken list after adding empty list (#19299) * Update lists.nim * Update tlists.nim * removed check `if b.tail != nil` The tail of the list being null it is still possible to retrieve its end by going through all nodes from the head. So checking for null from `b.tail` is unnecessary. However, setting `a.tail = b.tail` only if `a.head != nil`, so you don't break a good list with an already broken one. fixes #16617 [backport] (#19300) Update JS and nimscript import tests (#19306) * add new modules, except experimental ones * remove deprecated modules mersenne and sharedlist * better describe why some modules fail and some modules don't add compile time option for POSIX sigwait on Illumos/Solaris (#19296) * add compile time option for POSIX sigwait on Illumos/Solaris * fix link to documentation of `sigwait` on Illumos/Solaris [docs] clarify the raised exception (#19308) * [docs] clarify the raised exception Lest developers wanna know what the exception is. * Apply suggestions from @konsumlamm Co-authored-by: konsumlamm <[email protected]> Co-authored-by: konsumlamm <[email protected]> docs: Fix typo in tut1.rst (#19309) Fix #19314 - fixing broken `DoublyLinkedList` after adding empty `DoublyLinkedList` (#19315) [backport] * Update lists.nim * Update tlists.nim fixed typos (#19316) devel: style fix (#19318) this allows "--styleCheck:usages --styleCheck:error" docs: Fix typo in tut1.rst (#19324) correct the comments (#19322) --expandArc ``` var a b a = matrix(5, 5, 1.0) b = matrix(5, 5, 2.0) `=sink`(b, - let blitTmp = b wasMoved(b) blitTmp + a) `=destroy`(b) `=destroy`(a) ``` add std/private/win_getsysteminfo; refactor the usage of `GetSystemInfo` (#19310) * add std/private/win_getsysteminfo * import at the top level * wrappers follow nep1 too * follow review comment Update net.nim (#19327) [backport] Fix #19038 - making the Nim compiler work again on Windows XP (#19331) * Update osenv.nim * Update win_setenv.nim * Update lib/pure/includes/osenv.nim * Update lib/pure/includes/osenv.nim * fixing cstring Co-authored-by: Andreas Rumpf <[email protected]> fix nim-lang#19343 (#19344) [backport] Ensure HttpClient onProgress is called once per second Ensure that reported speed is accurate stylecheck usages part two: stdlib cleanup (#19338) * stylecheck usages part two: stdlib cleanup typeinfo.nim: importCompilerProc => importcompilerproc nre.nim: newLineFlags => newlineFlags system.nim: JSRoot => JsRoot ref #19319 * prefer importCompilerProc added filemode docs (#19346) Fix `remove` on last node of singly-linked list [backport:1.6] (#19353) fix stylecheck error with asyncdispatch (#19350) * stylecheck usages part two: stdlib cleanup typeinfo.nim: importCompilerProc => importcompilerproc nre.nim: newLineFlags => newlineFlags system.nim: JSRoot => JsRoot ref #19319 * prefer importCompilerProc * fix stylecheck error with asyncdispatch it is a partial regression since #12842 * add tests * don't use echo in tests remove spaces between an identifier and a star (#19355) It makes search easier by searching `+`* instead of `+` which filter lots of unexported versions. Follow nim-lang/Nim#18681 bitsets.nim: cleanup (#19361) make rst thread safe (#19369) split for the convenience of review docs: Mention `import foo {.all.}` syntax (#19377) Mention the `import foo {.all.}` syntax in the manual, with a caveat about private imports. Also link to the experimental importutils module. Co-authored-by: adigitoleo <[email protected]> update copyright year (#19381) docs: Fix broken cross references to `rfind` in strutils (#19382) [backport] Fixes three broken cross references to `rfind` in strutils. Breakage due to signature changes of the `rfind` methods. Co-authored-by: adigitoleo <[email protected]> move type operation section and remove deepcopy document (#19389) ref #19173; because deepcopy is not fit for ORC/ARC which was used for spawn and spawn will be removed from compiler deprecate unsafeAddr; extend addr (#19373) * deprecate unsafeAddr; extend addr addr is now available for all addressable locations, unsafeAddr is deprecated and become an alias for addr * follow @Vindaar's advice * change the signature of addr * unsafeAddr => addr (stdlib) * Update changelog.md * unsafeAddr => addr (tests) * Revert "unsafeAddr => addr (stdlib)" This reverts commit ab83c99c507048a8396e636bf22d55fdd84d7d1c. * doc changes; thanks to @konsumlamm Co-authored-by: konsumlamm <[email protected]> Co-authored-by: konsumlamm <[email protected]> improve changelog a bit (#19400) mangle names in nimbase.h using cppDefine (#19395) [backport] mangle names in nimbase.h fix comments Optimize lent in JS [backport:1.6] (#19393) * Optimize lent in JS [backport:1.6] * addr on lent doesn't work anymore, don't use it * use unsafeAddr in test again for older versions update deprecated example (#19415) `toNimIdent` proc is deprecated, so I replaced it with `ident` proc Improve Zshell completion (#19354) fix stricteffects (nimsuggest/sexp) (#19405) * fix stricteffects (nimsuggest/sexp) * Update tstrict_effects3.nim * Update tests/effects/tstrict_effects3.nim suppress deprecated warnings (#19408) * suppress deprecated warnings once bump version to 1.7.3 enable deprecated messages * deprecate later add an example to setControlCHook (#19416) * add an example to setControlCHook * [skip CI] format example for setControlCHook Co-authored-by: Nathan Blaxall <[email protected]> fix term rewriting with sideeffect (#19410) * fix term rewriting with sideeffect fix #6217 * add tests * Update tests/template/template_various.nim Resolve cross file resolution errors in atomics (#19422) [backport:1.6] * Resolve call undeclared routine testAndSet * Fix undeclared field atomicType Fix #11923 (#19427) * Apply commit nim-lang/Nim@5da931f that was never merged (was part of a bigger PR). Should fix issue #11932 * add a generic object for custom pragma os: faster getFileSize (#19438) Use "stat" rather than "open", "seek", and "close" system calls. The Windows implementation remains the same. bugfix: varargs count as open arrays (#19447) update outdated link (#19465) Ref nim-lang/Nim#19463 Update jsfetch with latest API and fix missing bindings (#19473) * Update with latest API and fix missing bindings remove deprecated `Body` remove implicit `cstring` convs add `Headers` to `FetchOptions` add `Request` init proc which takes `FetchOptions` * Update lib/std/jsfetch.nim Co-authored-by: Juan Carlos <[email protected]> * Update lib/std/jsfetch.nim Co-authored-by: Juan Carlos <[email protected]> * remove experimental flag Co-authored-by: Juan Carlos <[email protected]> No longer segfault when using a typeclass with a self referencing type (#19467) Clonkk fix2 11923 (#19451) * fix nnkBracketExpr not compiling for getImpl on customPragmaNode * fix test import * fix alias not working with hasCustomPragmas fix parseEnum cannot parse enum with const fields (#19466) fix #19463 Add compilers and hints to default nim.cfg (#18424) don't use a temp for addr [backport: 1.6] (#19503) * don't use a temp for addr fix #19497 * Update compiler/ccgcalls.nim Co-authored-by: konsumlamm <[email protected]> * add a test Co-authored-by: konsumlamm <[email protected]> fixes #19404 by protecting the memory we borrow from. this replaces crashes with minor memory leaks which seems to be acceptable. In the longer run we need a better VM that didn't grow hacks over a decade. (#19515) Co-authored-by: flywind <[email protected]> Remove backslash in glob pattern (#19524) use OrderedTable instead of OrderedTableRef for mimedb (#19522) * use OrderedTable instead of OrderedTableRef for mimedb Signed-off-by: David Krause <[email protected]> * added changelog entry for mimedb change Signed-off-by: David Krause <[email protected]> Remove Deprecated oids.oidsToString (#19519) * Remove deprecated oids.oidToString * Remove deprecated oids.oidToString Remove deprecated math.c_frexp (#19518) * Remove Deprecated math proc * Remove Deprecated math proc * Remove Deprecated math proc [testcase] genSym fails to make unique identifier for ref object types (#19506) close #15118 Documentation: Fix word usage (#19529) Update chcks.nim (#19540) keep casing of noinit and noreturn pragmas consistently documented (#19535) compile pragma: cache the result sooner (#19554) extccomp.addExternalFileToCompile() relies on hashes to decide whether an external C file needs recompilation or not. Due to short-circuit evaluation of boolean expressions, the procedure that generates a corresponding hash file is not called the first time an external file is compiled, so an avoidable recompilation is triggered the next build. This patch fixes that by moving the proc call with a desired side effect from its boolean expression, so it's executed unconditionally.

@konsumlamm

Concerns these changes: nim-lang/Nim@727c637...340b5a1 Excluded changes are: nim-lang/Nim#18963 nim-lang/Nim#19003 nim-lang/Nim#19043 nim-lang/Nim#19055 nim-lang/Nim#19053 nim-lang/Nim#19064 nim-lang/Nim#18642 nim-lang/Nim#19062 nim-lang/Nim#19082 nim-lang/Nim#19090 nim-lang/Nim#19077 nim-lang/Nim#19021 nim-lang/Nim#19100 nim-lang/Nim#19102 nim-lang/Nim#19111 nim-lang/Nim#19115 nim-lang/Nim#19133 nim-lang/Nim#19142 nim-lang/Nim#19158 nim-lang/Nim#19129 nim-lang/Nim#19137 nim-lang/Nim#19168 nim-lang/Nim#19156 nim-lang/Nim#19147 nim-lang/Nim#19180 nim-lang/Nim#19183 nim-lang/Nim#19182 nim-lang/Nim#19187 nim-lang/Nim#19179 nim-lang/Nim#19209 nim-lang/Nim#19210 nim-lang/Nim#19207 nim-lang/Nim#19219 nim-lang/Nim#19195 nim-lang/Nim#19212 nim-lang/Nim#19134 nim-lang/Nim#19235 nim-lang/Nim#19252 nim-lang/Nim#19196 nim-lang/Nim#19295 nim-lang/Nim#19301 nim-lang/Nim#19181 nim-lang/Nim#17223 nim-lang/Nim#19370 nim-lang/Nim#19385 nim-lang/Nim#19307 nim-lang/Nim#19394 nim-lang/Nim#19399 nim-lang/Nim#19390 nim-lang/Nim#19407 nim-lang/Nim#19419 nim-lang/Nim#19421 nim-lang/Nim#19363 nim-lang/Nim#19406 nim-lang/Nim#19431 nim-lang/Nim#19455 nim-lang/Nim#19461 nim-lang/Nim@cb894c7 nim-lang/Nim#19462 nim-lang/Nim#19442 nim-lang/Nim#19437 nim-lang/Nim#19433 nim-lang/Nim#19512 nim-lang/Nim#19487 nim-lang/Nim#19543 Excluded changes include major changes which require more consideration and changes which don't apply to the current code anymore but could be worth porting over still. Excluded changes which only change the identifier casing in tests or only concern code removed in nimskull aren't listed. Begin commit listing: use two underscores for easy demangling [backport:1.6] (#19028) Add Elbrus 2000 architecture (#19024) * Add Elbrus 2000 architecture * Add e2k to niminst * Update compiler/installer.ini Co-authored-by: Andreas Rumpf <[email protected]> remove exception (#18906) allow converting static vars to `openArray` (#19035) When assigning constant output to a seq, and then passing that static seq to other functions that take `openArray`, the compiler may end up producing errors, as it does not know how to convert `static[seq[T]]` to `openArray[T]`. By ignoring the `static` wrapper on the type for the purpose of determining data memory location and length, this gets resolved cleanly. Unfortunately, it is relatively tricky to come up with a minimal example, as there are followup problems from the failing conversion, e.g., this may lead to `internal error: inconsistent environment type`, instead of the relevant `openArrayLoc` error message. use the correct header for TIOCGWINSZ on Solaris (#19037) Minor update to terminal docs (#19056) * Update terminal.nim - Added some extra docs to cursorUp/Down/Forward/Backward - I was able to use hideCursor and showCursor without adding stdout, removed the parameter - Added docs to terminalHeight()* and terminalWidth()* * Update lib/pure/terminal.nim Co-authored-by: konsumlamm <[email protected]> * Update lib/pure/terminal.nim Co-authored-by: konsumlamm <[email protected]> * Added back f: file to cursor movement * Removed unnecessary comments Co-authored-by: konsumlamm <[email protected]> fix a tiny formating issue in doc/destructors.rst (#19058) fix a tiny code snippet formatting issue in `doc/constructors.rst`, again (#19065) Fix nimIdentNormalize, fixes #19067 (#19068) * Make nimIdentNormalize return "" when passed ""; fixes #19067 Fixes #19067 * Add tests for nimIdentNormalize fix #18971 (#19070) [backport:1.6] since the example code return value from global variable, instead of first argument, the `n.len` is 1 which causes compiler crashes. fixes #19000 (#19032) * fixes #19000 * progress fix #18410 (Errors initializing an object of RootObj with the C++ backend) [backport] (#18836) * fix #18410 * one line comment * typo * typo * cover cpp update numbers of lifetime-tracking hooks in doc/destructors.rst (#19088) bootstrapping Nim compiler with `cpp --gc:orc` (#19087) libs/impore/re: Add note about the requirement of `matches` to be pre-allocated (#19081) Add few runnableExamples for `findBounds` for clarity. Fixes nim-lang/Nim#18775 Add test for issue 15435 (#19079) * Add test for issue 15435 Closes nim-lang/Nim#15435. * Specify bug # in comment Addresses nim-lang/Nim#19079 (comment) manual: Document the use of `static` as a proc call (#19084) * manual: Document the use of `static` as a proc call Also adds tests. Fixes nim-lang/Nim#16987 . * Update doc/manual.rst Co-authored-by: konsumlamm <[email protected]> * Use the "bug #NNNN" comment syntax for consistency Ref: https://nim-lang.github.io/Nim/contributing.html#writing-tests-stdlib > Always refer to a GitHub issue using the following exact syntax: bug for tooling. * manual: Undocument usage of foo.static foo.static and foo.static() are not expected to work. Ref: https://github.com/nim-lang/Nim/pull/19084/files#r741203578 Co-authored-by: konsumlamm <[email protected]> manual: Document that comma propagates the default values of parameters (#19080) * manual: Document that comma propagates the default values of parameters Fixes nim-lang/Nim#15949. * Use the "bug #NNNN" comment syntax for consistency Ref: https://nim-lang.github.io/Nim/contributing.html#writing-tests-stdlib > Always refer to a GitHub issue using the following exact syntax: bug for tooling. fixes #19011 [backport:1.6] (#19114) Add deprecation pragmas in lib/deprecated/pure (#19113) Deprecate `std/sharedlist` and `std/sharedtables` (#19112) fix nimindexterm in rst2tex/doc2tex [backport] (#19106) * fix nimindexterm (rst2tex/doc2tex) [backport] * Add support for indexing in rst Call {.cursor.} a pragma. (#19116) * Call {.cursor.} a pragma. Its hard to find .curser annotation while googling because all other things like it are called pragmas. See https://nim-lang.org/docs/manual.html#pragmas Also the . in front of the name makes it hard to find and search for. Can we just call it cursor pragma? * Small fix for comment. Remove tlsEmulation enabled from Windows + GCC config (#19119) [backport:1.6] This flag has a very significant performance impact on programs compiled with --threads:on. It is also apparently not needed anymore for standard circumstances. Can we remove the config? See nim-lang/Nim#18146 (comment) for discussion and perf impact. [backport:1.6] Add security tip for setCookie (#19117) * Add security tip for setCookie * Update lib/pure/cookies.nim Co-authored-by: Dominik Picheta <[email protected]> * Update lib/pure/cookies.nim Co-authored-by: konsumlamm <[email protected]> Co-authored-by: Andreas Rumpf <[email protected]> Co-authored-by: Dominik Picheta <[email protected]> Co-authored-by: konsumlamm <[email protected]> correct cookie docs (#19122) refactoring: orc can use getThreadId() (#19123) * refactoring: orc can use getThreadId() * progress fixed colorNames sorting mistake (#19125) [backport] update manual (#19130) [backport] Merge file size fields correctly on Windows (#19141) * Merge file size fields correctly on Windows Merge file size fields correctly on Windows - Merge the two 32-bit file size fields from `BY_HANDLE_FILE_INFORMATION` correctly in `rawToFormalFileInfo`. - Fixes #19135 * Update os.nim Fix punycode.decode function (#19136) * Refactor: rename proc to func * Fix punycode.decode function This function could only properly decode punycodes containing a single encoded unicode character. As soon as there was more than one punycode character group to decode it produced invalid output - the number of characters was correct, but their position was not. * Update tpunycode.nim Co-authored-by: Clay Sweetser <[email protected]> Fix undeclared 'SYS_getrandom' on emscripten (#19144) wrong spaces (3 => 2) (#19145) `caseStmtMacros` no longer experimental, experimental manual refactor (#19173) * `caseStmtMacros` no longer experimental, experimental manual refactor * Update doc/manual.rst * apply review suggestions * apply review Co-authored-by: Andreas Rumpf <[email protected]> fix inline syntax highlighting in system.nim (#19184) swap port to correct port order (#19177) Co-authored-by: Jaremy Creechley <[email protected]> feat: TLS-ALPN wrappers for OpenSSL (#19202) Co-authored-by: Iced Quinn <[email protected]> misc bugfixes [backport:1.2] (#19203) treat do with pragmas but no parens as proc (#19191) fixes #19188 [format minor] remove unnecessary spaces (#19216) Making TCC work again on Windows --cpu:amd64 - fix #16326 (#19221) * fix #16326 * removing comments fixes a converter handling regression that caused private converters to leak into client modules; fixes #19213; [backport:1.6] (#19229) Add support for LoongArch (#19223) * Add support for LoongArch * Update compiler/installer.ini Co-authored-by: Andreas Rumpf <[email protected]> remove `std/sharedstrings` (#19228) * remove std/sharedstrings it has been broken since 0.18.0 * rephrase the changelog entry add comments to spawn and pinnedSpawn (#19230) `spawn` uses `nimSpawn3` internally and `pinnedSpawn` uses `nimSpawn4` internally. I comment it in order to help contributors get the gist of its functionality. fixes an old ARC bug: the produced copy/sink operations don't copy the hidden type field for objects with enabled inheritance; fixes #19205 [backport:1.6] (#19232) nimRawSetjmp: support Windows (#19197) * nimRawSetjmp: support Windows Using `_setjmp()` directly is required to avoid some rare (but very annoying) exception-related stack corruption leading to segfaults on Windows, with Mingw-w64 and SEH. More details: status-im/nimbus-eth2#3121 Also add "nimBuiltinSetjmp" - mostly for benchmarking. * fix for Apple's Clang++ Revert "swap port to correct port order (#19177)" (#19234) This reverts commit 0d0c249. move toDeque to after addLast (#19233) [backport:1.0] Changes the order of procs definitions in order to avoid calling an undefined proc. let Nim support Nimble 0.14 with lock-file support [backport:1.6] (#19236) nimc.rst: fix table markup (#19239) Various std net improvements (#19132) * Variant of that works with raw IpAddresses. - Add doc tests for new net proc's. - Aadd recvFrom impl - Add recvFrom impl -- tweak handling data var - Update lib/pure/net.nim Co-authored-by: Dominik Picheta <[email protected]> - cleaning up sendTo args - remove extra connect test - cleaning up sendTo args - fix inet_ntop test - fix test failing - byte len * fix test failing - byte len * debugging odd windows build failure * debugging odd windows build failure * more experiments to figure out the windows failure * try manual assigment on InAddr Co-authored-by: Jaremy Creechley <[email protected]> fix bug #14468 zero-width split (#19248) basicopt.txt: Unify the format (#19251) fix: fixes bug in CVerifyPeerUseEnvVars (#19247) Previously CVerifyPeerUseEnvVars was not being passed into scanSslCertificates, which meant that we weren't scanning additional certificate locations given via the SSL_CERT_FILE and SSL_CERT_DIR environment variables suggestion to respect typedarray type (#19257) * suggestion to respect typedarray * Update jssys.nim Co-authored-by: Sven Keller <[email protected]> fix #19244 - solves the problem of the InAddr object constructor in Windows. (#19259) * Update winlean.nim * Update tnet_ll.nim Fixed typo in manual.rst unsafeAssign->uncheckedAssign. Fixes part 1 of #19266 (#19267) use uppercase "type" for Proxy-Authorization header (#19273) Some servers will reject authorization requests with a lowercase "basic" type. Changing to "Basic" seems to solve these issues. https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Proxy-Authorization Update colors.nim (#19274) * Update colors.nim Added `lightgray` alias to `lightgrey` and `...grey`aliases for the rest of the gray colors. Added color `rebeccapurple`. Fixed the incorrect values for the `PaleVioletRed` and `MediumPurple` colors. This module should now be matching the CSS colors. I used the seq[tuple] syntax for defining the names. * Document colors changes. Extract runnables that specify `doccmd` (#19275) [backport:1.6] Fix build on FreeBSD/powerpc (#19282) It's currently misdetected as powerpc64. Fix #19107 (#19286) [backport] fixes grammar typos [backport] (#19289) fix 19292 (#19293) Fix #19297 - fixing broken list after adding empty list (#19299) * Update lists.nim * Update tlists.nim * removed check `if b.tail != nil` The tail of the list being null it is still possible to retrieve its end by going through all nodes from the head. So checking for null from `b.tail` is unnecessary. However, setting `a.tail = b.tail` only if `a.head != nil`, so you don't break a good list with an already broken one. fixes #16617 [backport] (#19300) Update JS and nimscript import tests (#19306) * add new modules, except experimental ones * remove deprecated modules mersenne and sharedlist * better describe why some modules fail and some modules don't add compile time option for POSIX sigwait on Illumos/Solaris (#19296) * add compile time option for POSIX sigwait on Illumos/Solaris * fix link to documentation of `sigwait` on Illumos/Solaris [docs] clarify the raised exception (#19308) * [docs] clarify the raised exception Lest developers wanna know what the exception is. * Apply suggestions from @konsumlamm Co-authored-by: konsumlamm <[email protected]> Co-authored-by: konsumlamm <[email protected]> docs: Fix typo in tut1.rst (#19309) Fix #19314 - fixing broken `DoublyLinkedList` after adding empty `DoublyLinkedList` (#19315) [backport] * Update lists.nim * Update tlists.nim fixed typos (#19316) devel: style fix (#19318) this allows "--styleCheck:usages --styleCheck:error" docs: Fix typo in tut1.rst (#19324) correct the comments (#19322) --expandArc ``` var a b a = matrix(5, 5, 1.0) b = matrix(5, 5, 2.0) `=sink`(b, - let blitTmp = b wasMoved(b) blitTmp + a) `=destroy`(b) `=destroy`(a) ``` add std/private/win_getsysteminfo; refactor the usage of `GetSystemInfo` (#19310) * add std/private/win_getsysteminfo * import at the top level * wrappers follow nep1 too * follow review comment Update net.nim (#19327) [backport] Fix #19038 - making the Nim compiler work again on Windows XP (#19331) * Update osenv.nim * Update win_setenv.nim * Update lib/pure/includes/osenv.nim * Update lib/pure/includes/osenv.nim * fixing cstring Co-authored-by: Andreas Rumpf <[email protected]> fix nim-lang#19343 (#19344) [backport] Ensure HttpClient onProgress is called once per second Ensure that reported speed is accurate stylecheck usages part two: stdlib cleanup (#19338) * stylecheck usages part two: stdlib cleanup typeinfo.nim: importCompilerProc => importcompilerproc nre.nim: newLineFlags => newlineFlags system.nim: JSRoot => JsRoot ref #19319 * prefer importCompilerProc added filemode docs (#19346) Fix `remove` on last node of singly-linked list [backport:1.6] (#19353) fix stylecheck error with asyncdispatch (#19350) * stylecheck usages part two: stdlib cleanup typeinfo.nim: importCompilerProc => importcompilerproc nre.nim: newLineFlags => newlineFlags system.nim: JSRoot => JsRoot ref #19319 * prefer importCompilerProc * fix stylecheck error with asyncdispatch it is a partial regression since #12842 * add tests * don't use echo in tests remove spaces between an identifier and a star (#19355) It makes search easier by searching `+`* instead of `+` which filter lots of unexported versions. Follow nim-lang/Nim#18681 bitsets.nim: cleanup (#19361) make rst thread safe (#19369) split for the convenience of review docs: Mention `import foo {.all.}` syntax (#19377) Mention the `import foo {.all.}` syntax in the manual, with a caveat about private imports. Also link to the experimental importutils module. Co-authored-by: adigitoleo <[email protected]> update copyright year (#19381) docs: Fix broken cross references to `rfind` in strutils (#19382) [backport] Fixes three broken cross references to `rfind` in strutils. Breakage due to signature changes of the `rfind` methods. Co-authored-by: adigitoleo <[email protected]> move type operation section and remove deepcopy document (#19389) ref #19173; because deepcopy is not fit for ORC/ARC which was used for spawn and spawn will be removed from compiler deprecate unsafeAddr; extend addr (#19373) * deprecate unsafeAddr; extend addr addr is now available for all addressable locations, unsafeAddr is deprecated and become an alias for addr * follow @Vindaar's advice * change the signature of addr * unsafeAddr => addr (stdlib) * Update changelog.md * unsafeAddr => addr (tests) * Revert "unsafeAddr => addr (stdlib)" This reverts commit ab83c99c507048a8396e636bf22d55fdd84d7d1c. * doc changes; thanks to @konsumlamm Co-authored-by: konsumlamm <[email protected]> Co-authored-by: konsumlamm <[email protected]> improve changelog a bit (#19400) mangle names in nimbase.h using cppDefine (#19395) [backport] mangle names in nimbase.h fix comments Optimize lent in JS [backport:1.6] (#19393) * Optimize lent in JS [backport:1.6] * addr on lent doesn't work anymore, don't use it * use unsafeAddr in test again for older versions update deprecated example (#19415) `toNimIdent` proc is deprecated, so I replaced it with `ident` proc Improve Zshell completion (#19354) fix stricteffects (nimsuggest/sexp) (#19405) * fix stricteffects (nimsuggest/sexp) * Update tstrict_effects3.nim * Update tests/effects/tstrict_effects3.nim suppress deprecated warnings (#19408) * suppress deprecated warnings once bump version to 1.7.3 enable deprecated messages * deprecate later add an example to setControlCHook (#19416) * add an example to setControlCHook * [skip CI] format example for setControlCHook Co-authored-by: Nathan Blaxall <[email protected]> fix term rewriting with sideeffect (#19410) * fix term rewriting with sideeffect fix #6217 * add tests * Update tests/template/template_various.nim Resolve cross file resolution errors in atomics (#19422) [backport:1.6] * Resolve call undeclared routine testAndSet * Fix undeclared field atomicType Fix #11923 (#19427) * Apply commit nim-lang/Nim@5da931f that was never merged (was part of a bigger PR). Should fix issue #11932 * add a generic object for custom pragma os: faster getFileSize (#19438) Use "stat" rather than "open", "seek", and "close" system calls. The Windows implementation remains the same. bugfix: varargs count as open arrays (#19447) update outdated link (#19465) Ref nim-lang/Nim#19463 Update jsfetch with latest API and fix missing bindings (#19473) * Update with latest API and fix missing bindings remove deprecated `Body` remove implicit `cstring` convs add `Headers` to `FetchOptions` add `Request` init proc which takes `FetchOptions` * Update lib/std/jsfetch.nim Co-authored-by: Juan Carlos <[email protected]> * Update lib/std/jsfetch.nim Co-authored-by: Juan Carlos <[email protected]> * remove experimental flag Co-authored-by: Juan Carlos <[email protected]> No longer segfault when using a typeclass with a self referencing type (#19467) Clonkk fix2 11923 (#19451) * fix nnkBracketExpr not compiling for getImpl on customPragmaNode * fix test import * fix alias not working with hasCustomPragmas fix parseEnum cannot parse enum with const fields (#19466) fix #19463 Add compilers and hints to default nim.cfg (#18424) don't use a temp for addr [backport: 1.6] (#19503) * don't use a temp for addr fix #19497 * Update compiler/ccgcalls.nim Co-authored-by: konsumlamm <[email protected]> * add a test Co-authored-by: konsumlamm <[email protected]> fixes #19404 by protecting the memory we borrow from. this replaces crashes with minor memory leaks which seems to be acceptable. In the longer run we need a better VM that didn't grow hacks over a decade. (#19515) Co-authored-by: flywind <[email protected]> Remove backslash in glob pattern (#19524) use OrderedTable instead of OrderedTableRef for mimedb (#19522) * use OrderedTable instead of OrderedTableRef for mimedb Signed-off-by: David Krause <[email protected]> * added changelog entry for mimedb change Signed-off-by: David Krause <[email protected]> Remove Deprecated oids.oidsToString (#19519) * Remove deprecated oids.oidToString * Remove deprecated oids.oidToString Remove deprecated math.c_frexp (#19518) * Remove Deprecated math proc * Remove Deprecated math proc * Remove Deprecated math proc [testcase] genSym fails to make unique identifier for ref object types (#19506) close #15118 Documentation: Fix word usage (#19529) Update chcks.nim (#19540) keep casing of noinit and noreturn pragmas consistently documented (#19535) compile pragma: cache the result sooner (#19554) extccomp.addExternalFileToCompile() relies on hashes to decide whether an external C file needs recompilation or not. Due to short-circuit evaluation of boolean expressions, the procedure that generates a corresponding hash file is not called the first time an external file is compiled, so an avoidable recompilation is triggered the next build. This patch fixes that by moving the proc call with a desired side effect from its boolean expression, so it's executed unconditionally.

@konsumlamm

Concerns these changes: nim-lang/Nim@727c637...340b5a1 Excluded changes are: nim-lang/Nim#18963 nim-lang/Nim#19003 nim-lang/Nim#19043 nim-lang/Nim#19055 nim-lang/Nim#19053 nim-lang/Nim#19064 nim-lang/Nim#18642 nim-lang/Nim#19062 nim-lang/Nim#19082 nim-lang/Nim#19090 nim-lang/Nim#19077 nim-lang/Nim#19021 nim-lang/Nim#19100 nim-lang/Nim#19102 nim-lang/Nim#19111 nim-lang/Nim#19115 nim-lang/Nim#19133 nim-lang/Nim#19142 nim-lang/Nim#19158 nim-lang/Nim#19129 nim-lang/Nim#19137 nim-lang/Nim#19168 nim-lang/Nim#19156 nim-lang/Nim#19147 nim-lang/Nim#19180 nim-lang/Nim#19183 nim-lang/Nim#19182 nim-lang/Nim#19187 nim-lang/Nim#19179 nim-lang/Nim#19209 nim-lang/Nim#19210 nim-lang/Nim#19207 nim-lang/Nim#19219 nim-lang/Nim#19195 nim-lang/Nim#19212 nim-lang/Nim#19134 nim-lang/Nim#19235 nim-lang/Nim#19252 nim-lang/Nim#19196 nim-lang/Nim#19295 nim-lang/Nim#19301 nim-lang/Nim#19181 nim-lang/Nim#17223 nim-lang/Nim#19370 nim-lang/Nim#19385 nim-lang/Nim#19307 nim-lang/Nim#19394 nim-lang/Nim#19399 nim-lang/Nim#19390 nim-lang/Nim#19407 nim-lang/Nim#19419 nim-lang/Nim#19421 nim-lang/Nim#19363 nim-lang/Nim#19406 nim-lang/Nim#19431 nim-lang/Nim#19455 nim-lang/Nim#19461 nim-lang/Nim@cb894c7 nim-lang/Nim#19462 nim-lang/Nim#19442 nim-lang/Nim#19437 nim-lang/Nim#19433 nim-lang/Nim#19512 nim-lang/Nim#19487 nim-lang/Nim#19543 Excluded changes include major changes which require more consideration and changes which don't apply to the current code anymore but could be worth porting over still. Excluded changes which only change the identifier casing in tests or only concern code removed in nimskull aren't listed. Begin commit listing: use two underscores for easy demangling [backport:1.6] (#19028) Add Elbrus 2000 architecture (#19024) * Add Elbrus 2000 architecture * Add e2k to niminst * Update compiler/installer.ini Co-authored-by: Andreas Rumpf <[email protected]> remove exception (#18906) allow converting static vars to `openArray` (#19035) When assigning constant output to a seq, and then passing that static seq to other functions that take `openArray`, the compiler may end up producing errors, as it does not know how to convert `static[seq[T]]` to `openArray[T]`. By ignoring the `static` wrapper on the type for the purpose of determining data memory location and length, this gets resolved cleanly. Unfortunately, it is relatively tricky to come up with a minimal example, as there are followup problems from the failing conversion, e.g., this may lead to `internal error: inconsistent environment type`, instead of the relevant `openArrayLoc` error message. use the correct header for TIOCGWINSZ on Solaris (#19037) Minor update to terminal docs (#19056) * Update terminal.nim - Added some extra docs to cursorUp/Down/Forward/Backward - I was able to use hideCursor and showCursor without adding stdout, removed the parameter - Added docs to terminalHeight()* and terminalWidth()* * Update lib/pure/terminal.nim Co-authored-by: konsumlamm <[email protected]> * Update lib/pure/terminal.nim Co-authored-by: konsumlamm <[email protected]> * Added back f: file to cursor movement * Removed unnecessary comments Co-authored-by: konsumlamm <[email protected]> fix a tiny formating issue in doc/destructors.rst (#19058) fix a tiny code snippet formatting issue in `doc/constructors.rst`, again (#19065) Fix nimIdentNormalize, fixes #19067 (#19068) * Make nimIdentNormalize return "" when passed ""; fixes #19067 Fixes #19067 * Add tests for nimIdentNormalize fix #18971 (#19070) [backport:1.6] since the example code return value from global variable, instead of first argument, the `n.len` is 1 which causes compiler crashes. fixes #19000 (#19032) * fixes #19000 * progress fix #18410 (Errors initializing an object of RootObj with the C++ backend) [backport] (#18836) * fix #18410 * one line comment * typo * typo * cover cpp update numbers of lifetime-tracking hooks in doc/destructors.rst (#19088) bootstrapping Nim compiler with `cpp --gc:orc` (#19087) libs/impore/re: Add note about the requirement of `matches` to be pre-allocated (#19081) Add few runnableExamples for `findBounds` for clarity. Fixes nim-lang/Nim#18775 Add test for issue 15435 (#19079) * Add test for issue 15435 Closes nim-lang/Nim#15435. * Specify bug # in comment Addresses nim-lang/Nim#19079 (comment) manual: Document the use of `static` as a proc call (#19084) * manual: Document the use of `static` as a proc call Also adds tests. Fixes nim-lang/Nim#16987 . * Update doc/manual.rst Co-authored-by: konsumlamm <[email protected]> * Use the "bug #NNNN" comment syntax for consistency Ref: https://nim-lang.github.io/Nim/contributing.html#writing-tests-stdlib > Always refer to a GitHub issue using the following exact syntax: bug for tooling. * manual: Undocument usage of foo.static foo.static and foo.static() are not expected to work. Ref: https://github.com/nim-lang/Nim/pull/19084/files#r741203578 Co-authored-by: konsumlamm <[email protected]> manual: Document that comma propagates the default values of parameters (#19080) * manual: Document that comma propagates the default values of parameters Fixes nim-lang/Nim#15949. * Use the "bug #NNNN" comment syntax for consistency Ref: https://nim-lang.github.io/Nim/contributing.html#writing-tests-stdlib > Always refer to a GitHub issue using the following exact syntax: bug for tooling. fixes #19011 [backport:1.6] (#19114) Add deprecation pragmas in lib/deprecated/pure (#19113) Deprecate `std/sharedlist` and `std/sharedtables` (#19112) fix nimindexterm in rst2tex/doc2tex [backport] (#19106) * fix nimindexterm (rst2tex/doc2tex) [backport] * Add support for indexing in rst Call {.cursor.} a pragma. (#19116) * Call {.cursor.} a pragma. Its hard to find .curser annotation while googling because all other things like it are called pragmas. See https://nim-lang.org/docs/manual.html#pragmas Also the . in front of the name makes it hard to find and search for. Can we just call it cursor pragma? * Small fix for comment. Remove tlsEmulation enabled from Windows + GCC config (#19119) [backport:1.6] This flag has a very significant performance impact on programs compiled with --threads:on. It is also apparently not needed anymore for standard circumstances. Can we remove the config? See nim-lang/Nim#18146 (comment) for discussion and perf impact. [backport:1.6] Add security tip for setCookie (#19117) * Add security tip for setCookie * Update lib/pure/cookies.nim Co-authored-by: Dominik Picheta <[email protected]> * Update lib/pure/cookies.nim Co-authored-by: konsumlamm <[email protected]> Co-authored-by: Andreas Rumpf <[email protected]> Co-authored-by: Dominik Picheta <[email protected]> Co-authored-by: konsumlamm <[email protected]> correct cookie docs (#19122) refactoring: orc can use getThreadId() (#19123) * refactoring: orc can use getThreadId() * progress fixed colorNames sorting mistake (#19125) [backport] update manual (#19130) [backport] Merge file size fields correctly on Windows (#19141) * Merge file size fields correctly on Windows Merge file size fields correctly on Windows - Merge the two 32-bit file size fields from `BY_HANDLE_FILE_INFORMATION` correctly in `rawToFormalFileInfo`. - Fixes #19135 * Update os.nim Fix punycode.decode function (#19136) * Refactor: rename proc to func * Fix punycode.decode function This function could only properly decode punycodes containing a single encoded unicode character. As soon as there was more than one punycode character group to decode it produced invalid output - the number of characters was correct, but their position was not. * Update tpunycode.nim Co-authored-by: Clay Sweetser <[email protected]> Fix undeclared 'SYS_getrandom' on emscripten (#19144) wrong spaces (3 => 2) (#19145) `caseStmtMacros` no longer experimental, experimental manual refactor (#19173) * `caseStmtMacros` no longer experimental, experimental manual refactor * Update doc/manual.rst * apply review suggestions * apply review Co-authored-by: Andreas Rumpf <[email protected]> fix inline syntax highlighting in system.nim (#19184) swap port to correct port order (#19177) Co-authored-by: Jaremy Creechley <[email protected]> feat: TLS-ALPN wrappers for OpenSSL (#19202) Co-authored-by: Iced Quinn <[email protected]> misc bugfixes [backport:1.2] (#19203) treat do with pragmas but no parens as proc (#19191) fixes #19188 [format minor] remove unnecessary spaces (#19216) Making TCC work again on Windows --cpu:amd64 - fix #16326 (#19221) * fix #16326 * removing comments fixes a converter handling regression that caused private converters to leak into client modules; fixes #19213; [backport:1.6] (#19229) Add support for LoongArch (#19223) * Add support for LoongArch * Update compiler/installer.ini Co-authored-by: Andreas Rumpf <[email protected]> remove `std/sharedstrings` (#19228) * remove std/sharedstrings it has been broken since 0.18.0 * rephrase the changelog entry add comments to spawn and pinnedSpawn (#19230) `spawn` uses `nimSpawn3` internally and `pinnedSpawn` uses `nimSpawn4` internally. I comment it in order to help contributors get the gist of its functionality. fixes an old ARC bug: the produced copy/sink operations don't copy the hidden type field for objects with enabled inheritance; fixes #19205 [backport:1.6] (#19232) nimRawSetjmp: support Windows (#19197) * nimRawSetjmp: support Windows Using `_setjmp()` directly is required to avoid some rare (but very annoying) exception-related stack corruption leading to segfaults on Windows, with Mingw-w64 and SEH. More details: status-im/nimbus-eth2#3121 Also add "nimBuiltinSetjmp" - mostly for benchmarking. * fix for Apple's Clang++ Revert "swap port to correct port order (#19177)" (#19234) This reverts commit 0d0c249. move toDeque to after addLast (#19233) [backport:1.0] Changes the order of procs definitions in order to avoid calling an undefined proc. let Nim support Nimble 0.14 with lock-file support [backport:1.6] (#19236) nimc.rst: fix table markup (#19239) Various std net improvements (#19132) * Variant of that works with raw IpAddresses. - Add doc tests for new net proc's. - Aadd recvFrom impl - Add recvFrom impl -- tweak handling data var - Update lib/pure/net.nim Co-authored-by: Dominik Picheta <[email protected]> - cleaning up sendTo args - remove extra connect test - cleaning up sendTo args - fix inet_ntop test - fix test failing - byte len * fix test failing - byte len * debugging odd windows build failure * debugging odd windows build failure * more experiments to figure out the windows failure * try manual assigment on InAddr Co-authored-by: Jaremy Creechley <[email protected]> fix bug #14468 zero-width split (#19248) basicopt.txt: Unify the format (#19251) fix: fixes bug in CVerifyPeerUseEnvVars (#19247) Previously CVerifyPeerUseEnvVars was not being passed into scanSslCertificates, which meant that we weren't scanning additional certificate locations given via the SSL_CERT_FILE and SSL_CERT_DIR environment variables suggestion to respect typedarray type (#19257) * suggestion to respect typedarray * Update jssys.nim Co-authored-by: Sven Keller <[email protected]> fix #19244 - solves the problem of the InAddr object constructor in Windows. (#19259) * Update winlean.nim * Update tnet_ll.nim Fixed typo in manual.rst unsafeAssign->uncheckedAssign. Fixes part 1 of #19266 (#19267) use uppercase "type" for Proxy-Authorization header (#19273) Some servers will reject authorization requests with a lowercase "basic" type. Changing to "Basic" seems to solve these issues. https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Proxy-Authorization Update colors.nim (#19274) * Update colors.nim Added `lightgray` alias to `lightgrey` and `...grey`aliases for the rest of the gray colors. Added color `rebeccapurple`. Fixed the incorrect values for the `PaleVioletRed` and `MediumPurple` colors. This module should now be matching the CSS colors. I used the seq[tuple] syntax for defining the names. * Document colors changes. Extract runnables that specify `doccmd` (#19275) [backport:1.6] Fix build on FreeBSD/powerpc (#19282) It's currently misdetected as powerpc64. Fix #19107 (#19286) [backport] fixes grammar typos [backport] (#19289) fix 19292 (#19293) Fix #19297 - fixing broken list after adding empty list (#19299) * Update lists.nim * Update tlists.nim * removed check `if b.tail != nil` The tail of the list being null it is still possible to retrieve its end by going through all nodes from the head. So checking for null from `b.tail` is unnecessary. However, setting `a.tail = b.tail` only if `a.head != nil`, so you don't break a good list with an already broken one. fixes #16617 [backport] (#19300) Update JS and nimscript import tests (#19306) * add new modules, except experimental ones * remove deprecated modules mersenne and sharedlist * better describe why some modules fail and some modules don't add compile time option for POSIX sigwait on Illumos/Solaris (#19296) * add compile time option for POSIX sigwait on Illumos/Solaris * fix link to documentation of `sigwait` on Illumos/Solaris [docs] clarify the raised exception (#19308) * [docs] clarify the raised exception Lest developers wanna know what the exception is. * Apply suggestions from @konsumlamm Co-authored-by: konsumlamm <[email protected]> Co-authored-by: konsumlamm <[email protected]> docs: Fix typo in tut1.rst (#19309) Fix #19314 - fixing broken `DoublyLinkedList` after adding empty `DoublyLinkedList` (#19315) [backport] * Update lists.nim * Update tlists.nim fixed typos (#19316) devel: style fix (#19318) this allows "--styleCheck:usages --styleCheck:error" docs: Fix typo in tut1.rst (#19324) correct the comments (#19322) --expandArc ``` var a b a = matrix(5, 5, 1.0) b = matrix(5, 5, 2.0) `=sink`(b, - let blitTmp = b wasMoved(b) blitTmp + a) `=destroy`(b) `=destroy`(a) ``` add std/private/win_getsysteminfo; refactor the usage of `GetSystemInfo` (#19310) * add std/private/win_getsysteminfo * import at the top level * wrappers follow nep1 too * follow review comment Update net.nim (#19327) [backport] Fix #19038 - making the Nim compiler work again on Windows XP (#19331) * Update osenv.nim * Update win_setenv.nim * Update lib/pure/includes/osenv.nim * Update lib/pure/includes/osenv.nim * fixing cstring Co-authored-by: Andreas Rumpf <[email protected]> fix nim-lang#19343 (#19344) [backport] Ensure HttpClient onProgress is called once per second Ensure that reported speed is accurate stylecheck usages part two: stdlib cleanup (#19338) * stylecheck usages part two: stdlib cleanup typeinfo.nim: importCompilerProc => importcompilerproc nre.nim: newLineFlags => newlineFlags system.nim: JSRoot => JsRoot ref #19319 * prefer importCompilerProc added filemode docs (#19346) Fix `remove` on last node of singly-linked list [backport:1.6] (#19353) fix stylecheck error with asyncdispatch (#19350) * stylecheck usages part two: stdlib cleanup typeinfo.nim: importCompilerProc => importcompilerproc nre.nim: newLineFlags => newlineFlags system.nim: JSRoot => JsRoot ref #19319 * prefer importCompilerProc * fix stylecheck error with asyncdispatch it is a partial regression since #12842 * add tests * don't use echo in tests remove spaces between an identifier and a star (#19355) It makes search easier by searching `+`* instead of `+` which filter lots of unexported versions. Follow nim-lang/Nim#18681 bitsets.nim: cleanup (#19361) make rst thread safe (#19369) split for the convenience of review docs: Mention `import foo {.all.}` syntax (#19377) Mention the `import foo {.all.}` syntax in the manual, with a caveat about private imports. Also link to the experimental importutils module. Co-authored-by: adigitoleo <[email protected]> update copyright year (#19381) docs: Fix broken cross references to `rfind` in strutils (#19382) [backport] Fixes three broken cross references to `rfind` in strutils. Breakage due to signature changes of the `rfind` methods. Co-authored-by: adigitoleo <[email protected]> move type operation section and remove deepcopy document (#19389) ref #19173; because deepcopy is not fit for ORC/ARC which was used for spawn and spawn will be removed from compiler deprecate unsafeAddr; extend addr (#19373) * deprecate unsafeAddr; extend addr addr is now available for all addressable locations, unsafeAddr is deprecated and become an alias for addr * follow @Vindaar's advice * change the signature of addr * unsafeAddr => addr (stdlib) * Update changelog.md * unsafeAddr => addr (tests) * Revert "unsafeAddr => addr (stdlib)" This reverts commit ab83c99c507048a8396e636bf22d55fdd84d7d1c. * doc changes; thanks to @konsumlamm Co-authored-by: konsumlamm <[email protected]> Co-authored-by: konsumlamm <[email protected]> improve changelog a bit (#19400) mangle names in nimbase.h using cppDefine (#19395) [backport] mangle names in nimbase.h fix comments Optimize lent in JS [backport:1.6] (#19393) * Optimize lent in JS [backport:1.6] * addr on lent doesn't work anymore, don't use it * use unsafeAddr in test again for older versions update deprecated example (#19415) `toNimIdent` proc is deprecated, so I replaced it with `ident` proc Improve Zshell completion (#19354) fix stricteffects (nimsuggest/sexp) (#19405) * fix stricteffects (nimsuggest/sexp) * Update tstrict_effects3.nim * Update tests/effects/tstrict_effects3.nim suppress deprecated warnings (#19408) * suppress deprecated warnings once bump version to 1.7.3 enable deprecated messages * deprecate later add an example to setControlCHook (#19416) * add an example to setControlCHook * [skip CI] format example for setControlCHook Co-authored-by: Nathan Blaxall <[email protected]> fix term rewriting with sideeffect (#19410) * fix term rewriting with sideeffect fix #6217 * add tests * Update tests/template/template_various.nim Resolve cross file resolution errors in atomics (#19422) [backport:1.6] * Resolve call undeclared routine testAndSet * Fix undeclared field atomicType Fix #11923 (#19427) * Apply commit nim-lang/Nim@5da931f that was never merged (was part of a bigger PR). Should fix issue #11932 * add a generic object for custom pragma os: faster getFileSize (#19438) Use "stat" rather than "open", "seek", and "close" system calls. The Windows implementation remains the same. bugfix: varargs count as open arrays (#19447) update outdated link (#19465) Ref nim-lang/Nim#19463 Update jsfetch with latest API and fix missing bindings (#19473) * Update with latest API and fix missing bindings remove deprecated `Body` remove implicit `cstring` convs add `Headers` to `FetchOptions` add `Request` init proc which takes `FetchOptions` * Update lib/std/jsfetch.nim Co-authored-by: Juan Carlos <[email protected]> * Update lib/std/jsfetch.nim Co-authored-by: Juan Carlos <[email protected]> * remove experimental flag Co-authored-by: Juan Carlos <[email protected]> No longer segfault when using a typeclass with a self referencing type (#19467) Clonkk fix2 11923 (#19451) * fix nnkBracketExpr not compiling for getImpl on customPragmaNode * fix test import * fix alias not working with hasCustomPragmas fix parseEnum cannot parse enum with const fields (#19466) fix #19463 Add compilers and hints to default nim.cfg (#18424) don't use a temp for addr [backport: 1.6] (#19503) * don't use a temp for addr fix #19497 * Update compiler/ccgcalls.nim Co-authored-by: konsumlamm <[email protected]> * add a test Co-authored-by: konsumlamm <[email protected]> fixes #19404 by protecting the memory we borrow from. this replaces crashes with minor memory leaks which seems to be acceptable. In the longer run we need a better VM that didn't grow hacks over a decade. (#19515) Co-authored-by: flywind <[email protected]> Remove backslash in glob pattern (#19524) use OrderedTable instead of OrderedTableRef for mimedb (#19522) * use OrderedTable instead of OrderedTableRef for mimedb Signed-off-by: David Krause <[email protected]> * added changelog entry for mimedb change Signed-off-by: David Krause <[email protected]> Remove Deprecated oids.oidsToString (#19519) * Remove deprecated oids.oidToString * Remove deprecated oids.oidToString Remove deprecated math.c_frexp (#19518) * Remove Deprecated math proc * Remove Deprecated math proc * Remove Deprecated math proc [testcase] genSym fails to make unique identifier for ref object types (#19506) close #15118 Documentation: Fix word usage (#19529) Update chcks.nim (#19540) keep casing of noinit and noreturn pragmas consistently documented (#19535) compile pragma: cache the result sooner (#19554) extccomp.addExternalFileToCompile() relies on hashes to decide whether an external C file needs recompilation or not. Due to short-circuit evaluation of boolean expressions, the procedure that generates a corresponding hash file is not called the first time an external file is compiled, so an avoidable recompilation is triggered the next build. This patch fixes that by moving the proc call with a desired side effect from its boolean expression, so it's executed unconditionally.

240: Merge upstream changes r=Clyybber a=Clyybber Concerns these changes: nim-lang/Nim@727c637...340b5a1 Excluded changes are: nim-lang/Nim#18963 nim-lang/Nim#19003 nim-lang/Nim#19043 nim-lang/Nim#19055 nim-lang/Nim#19053 nim-lang/Nim#19064 nim-lang/Nim#18642 nim-lang/Nim#19062 nim-lang/Nim#19082 nim-lang/Nim#19090 nim-lang/Nim#19077 nim-lang/Nim#19021 nim-lang/Nim#19100 nim-lang/Nim#19102 nim-lang/Nim#19111 nim-lang/Nim#19115 nim-lang/Nim#19133 nim-lang/Nim#19142 nim-lang/Nim#19158 nim-lang/Nim#19129 nim-lang/Nim#19137 nim-lang/Nim#19168 nim-lang/Nim#19156 nim-lang/Nim#19147 nim-lang/Nim#19180 nim-lang/Nim#19183 nim-lang/Nim#19182 nim-lang/Nim#19187 nim-lang/Nim#19179 nim-lang/Nim#19209 nim-lang/Nim#19210 nim-lang/Nim#19207 nim-lang/Nim#19219 nim-lang/Nim#19195 nim-lang/Nim#19212 nim-lang/Nim#19134 nim-lang/Nim#19235 nim-lang/Nim#19252 nim-lang/Nim#19196 nim-lang/Nim#19295 nim-lang/Nim#19301 nim-lang/Nim#19181 nim-lang/Nim#17223 nim-lang/Nim#19370 nim-lang/Nim#19385 nim-lang/Nim#19307 nim-lang/Nim#19394 nim-lang/Nim#19399 nim-lang/Nim#19390 nim-lang/Nim#19407 nim-lang/Nim#19419 nim-lang/Nim#19421 nim-lang/Nim#19363 nim-lang/Nim#19406 nim-lang/Nim#19431 nim-lang/Nim#19455 nim-lang/Nim#19461 nim-lang/Nim@cb894c7 nim-lang/Nim#19462 nim-lang/Nim#19442 nim-lang/Nim#19437 nim-lang/Nim#19433 nim-lang/Nim#19512 nim-lang/Nim#19487 nim-lang/Nim#19543 Excluded changes include major changes which require more consideration and changes which don't apply to the current code anymore but could be worth porting over still. Excluded changes which only change the identifier casing in tests or only concern code removed in nimskull aren't listed. Commit listing is in the commit message Co-authored-by: The Nim Contributors <>

* Porting Nim to run on Zephyr. Includes changes to `std/net`. Squashed commit of the following: tweaking more memory / malloc things revert back bitmasks tweaking nim to use kernel heap as C malloc doesn't work fixing socket polling on zephyr cleanup getting maximum sockets for process or for rtos'es reorganizing and fixing net for async / system merge netlite changes back into nativesockets merge netlite changes back into nativesockets reverting native sockets back tweaking nim / zephyr network adding option to run 'net-lite' from linux bridging zephyr's max connections fixing net errors fixing compilation with getAddrString fixing compilation with getAddrString experimenting with a nativesockets_lite ... getAddrString experimenting with a nativesockets_lite ... getAddrString experimenting with a nativesockets_lite ... getLocalAddr experimenting with a nativesockets_lite ... getLocalAddr experimenting with a nativesockets_lite ... add note regarding incorrect FreeRTOS Sockadd_in fields changing to NIM_STATIC_ASSERT cleaning up the static_assert error messages cleaning up the static_assert error messages setting up static assert ftw! testing compile time asserts reworking Sockaddr objects to more closely match various platforms reworking Sockaddr objects to more closely match various platforms reworking Sockaddr objects to more closely match various platforms finding missing items (issue #18684) fixup posix constants (issue #18684) adding plumbing for zephyr os (issue #18684) adding plumbing for zephyr os (issue #18684) * fixing constant capitalizations * remove extra debug prints and fix TSa_Family/cint issue * remove extra debug prints and fix TSa_Family/cint issue * Porting Nim to run on Zephyr. Includes changes to `std/net`. Squashed commit of the following: tweaking more memory / malloc things revert back bitmasks tweaking nim to use kernel heap as C malloc doesn't work fixing socket polling on zephyr cleanup getting maximum sockets for process or for rtos'es reorganizing and fixing net for async / system merge netlite changes back into nativesockets merge netlite changes back into nativesockets reverting native sockets back tweaking nim / zephyr network adding option to run 'net-lite' from linux bridging zephyr's max connections fixing net errors fixing compilation with getAddrString fixing compilation with getAddrString experimenting with a nativesockets_lite ... getAddrString experimenting with a nativesockets_lite ... getAddrString experimenting with a nativesockets_lite ... getLocalAddr experimenting with a nativesockets_lite ... getLocalAddr experimenting with a nativesockets_lite ... add note regarding incorrect FreeRTOS Sockadd_in fields changing to NIM_STATIC_ASSERT cleaning up the static_assert error messages cleaning up the static_assert error messages setting up static assert ftw! testing compile time asserts reworking Sockaddr objects to more closely match various platforms reworking Sockaddr objects to more closely match various platforms reworking Sockaddr objects to more closely match various platforms finding missing items (issue #18684) fixup posix constants (issue #18684) adding plumbing for zephyr os (issue #18684) adding plumbing for zephyr os (issue #18684) * fixing constant capitalizations * remove extra debug prints and fix TSa_Family/cint issue * remove extra debug prints and fix TSa_Family/cint issue * fixing PR issues * Porting Nim to run on Zephyr. Includes changes to `std/net`. Squashed commit of the following: tweaking more memory / malloc things revert back bitmasks tweaking nim to use kernel heap as C malloc doesn't work fixing socket polling on zephyr cleanup getting maximum sockets for process or for rtos'es reorganizing and fixing net for async / system merge netlite changes back into nativesockets merge netlite changes back into nativesockets reverting native sockets back tweaking nim / zephyr network adding option to run 'net-lite' from linux bridging zephyr's max connections fixing net errors fixing compilation with getAddrString fixing compilation with getAddrString experimenting with a nativesockets_lite ... getAddrString experimenting with a nativesockets_lite ... getAddrString experimenting with a nativesockets_lite ... getLocalAddr experimenting with a nativesockets_lite ... getLocalAddr experimenting with a nativesockets_lite ... add note regarding incorrect FreeRTOS Sockadd_in fields changing to NIM_STATIC_ASSERT cleaning up the static_assert error messages cleaning up the static_assert error messages setting up static assert ftw! testing compile time asserts reworking Sockaddr objects to more closely match various platforms reworking Sockaddr objects to more closely match various platforms reworking Sockaddr objects to more closely match various platforms finding missing items (issue #18684) fixup posix constants (issue #18684) adding plumbing for zephyr os (issue #18684) adding plumbing for zephyr os (issue #18684) * fixing constant capitalizations * remove extra debug prints and fix TSa_Family/cint issue * remove extra debug prints and fix TSa_Family/cint issue * Remerge * fixing constant capitalizations * remove extra debug prints and fix TSa_Family/cint issue * remove extra debug prints and fix TSa_Family/cint issue * fixing PR issues * fix maxDescriptors on zephyr/freertos * move maxDescriptors to selector.nim -- fixes compile issue * change realloc impl on zephyr to match ansi c behavior * change realloc impl on zephyr to match ansi c behavior * force compileOnly mode for tlwip Co-authored-by: Jaremy J. Creechley <[email protected]> Co-authored-by: Jaremy Creechley <[email protected]> (cherry picked from commit 141b76e)

* Porting Nim to run on Zephyr. Includes changes to `std/net`. Squashed commit of the following: tweaking more memory / malloc things revert back bitmasks tweaking nim to use kernel heap as C malloc doesn't work fixing socket polling on zephyr cleanup getting maximum sockets for process or for rtos'es reorganizing and fixing net for async / system merge netlite changes back into nativesockets merge netlite changes back into nativesockets reverting native sockets back tweaking nim / zephyr network adding option to run 'net-lite' from linux bridging zephyr's max connections fixing net errors fixing compilation with getAddrString fixing compilation with getAddrString experimenting with a nativesockets_lite ... getAddrString experimenting with a nativesockets_lite ... getAddrString experimenting with a nativesockets_lite ... getLocalAddr experimenting with a nativesockets_lite ... getLocalAddr experimenting with a nativesockets_lite ... add note regarding incorrect FreeRTOS Sockadd_in fields changing to NIM_STATIC_ASSERT cleaning up the static_assert error messages cleaning up the static_assert error messages setting up static assert ftw! testing compile time asserts reworking Sockaddr objects to more closely match various platforms reworking Sockaddr objects to more closely match various platforms reworking Sockaddr objects to more closely match various platforms finding missing items (issue nim-lang#18684) fixup posix constants (issue nim-lang#18684) adding plumbing for zephyr os (issue nim-lang#18684) adding plumbing for zephyr os (issue nim-lang#18684) * fixing constant capitalizations * remove extra debug prints and fix TSa_Family/cint issue * remove extra debug prints and fix TSa_Family/cint issue * Porting Nim to run on Zephyr. Includes changes to `std/net`. Squashed commit of the following: tweaking more memory / malloc things revert back bitmasks tweaking nim to use kernel heap as C malloc doesn't work fixing socket polling on zephyr cleanup getting maximum sockets for process or for rtos'es reorganizing and fixing net for async / system merge netlite changes back into nativesockets merge netlite changes back into nativesockets reverting native sockets back tweaking nim / zephyr network adding option to run 'net-lite' from linux bridging zephyr's max connections fixing net errors fixing compilation with getAddrString fixing compilation with getAddrString experimenting with a nativesockets_lite ... getAddrString experimenting with a nativesockets_lite ... getAddrString experimenting with a nativesockets_lite ... getLocalAddr experimenting with a nativesockets_lite ... getLocalAddr experimenting with a nativesockets_lite ... add note regarding incorrect FreeRTOS Sockadd_in fields changing to NIM_STATIC_ASSERT cleaning up the static_assert error messages cleaning up the static_assert error messages setting up static assert ftw! testing compile time asserts reworking Sockaddr objects to more closely match various platforms reworking Sockaddr objects to more closely match various platforms reworking Sockaddr objects to more closely match various platforms finding missing items (issue nim-lang#18684) fixup posix constants (issue nim-lang#18684) adding plumbing for zephyr os (issue nim-lang#18684) adding plumbing for zephyr os (issue nim-lang#18684) * fixing constant capitalizations * remove extra debug prints and fix TSa_Family/cint issue * remove extra debug prints and fix TSa_Family/cint issue * fixing PR issues * Porting Nim to run on Zephyr. Includes changes to `std/net`. Squashed commit of the following: tweaking more memory / malloc things revert back bitmasks tweaking nim to use kernel heap as C malloc doesn't work fixing socket polling on zephyr cleanup getting maximum sockets for process or for rtos'es reorganizing and fixing net for async / system merge netlite changes back into nativesockets merge netlite changes back into nativesockets reverting native sockets back tweaking nim / zephyr network adding option to run 'net-lite' from linux bridging zephyr's max connections fixing net errors fixing compilation with getAddrString fixing compilation with getAddrString experimenting with a nativesockets_lite ... getAddrString experimenting with a nativesockets_lite ... getAddrString experimenting with a nativesockets_lite ... getLocalAddr experimenting with a nativesockets_lite ... getLocalAddr experimenting with a nativesockets_lite ... add note regarding incorrect FreeRTOS Sockadd_in fields changing to NIM_STATIC_ASSERT cleaning up the static_assert error messages cleaning up the static_assert error messages setting up static assert ftw! testing compile time asserts reworking Sockaddr objects to more closely match various platforms reworking Sockaddr objects to more closely match various platforms reworking Sockaddr objects to more closely match various platforms finding missing items (issue nim-lang#18684) fixup posix constants (issue nim-lang#18684) adding plumbing for zephyr os (issue nim-lang#18684) adding plumbing for zephyr os (issue nim-lang#18684) * fixing constant capitalizations * remove extra debug prints and fix TSa_Family/cint issue * remove extra debug prints and fix TSa_Family/cint issue * Remerge * fixing constant capitalizations * remove extra debug prints and fix TSa_Family/cint issue * remove extra debug prints and fix TSa_Family/cint issue * fixing PR issues * fix maxDescriptors on zephyr/freertos * move maxDescriptors to selector.nim -- fixes compile issue * change realloc impl on zephyr to match ansi c behavior * change realloc impl on zephyr to match ansi c behavior * force compileOnly mode for tlwip Co-authored-by: Jaremy J. Creechley <[email protected]> Co-authored-by: Jaremy Creechley <[email protected]>

elcritch commented Oct 16, 2021

View reviewed changes

Jaremy J. Creechley added 3 commits October 15, 2021 19:59

fixing constant capitalizations

92e1952

remove extra debug prints and fix TSa_Family/cint issue

03246dd

remove extra debug prints and fix TSa_Family/cint issue

6d61911

Araq reviewed Oct 16, 2021

View reviewed changes

Jaremy J. Creechley and others added 16 commits October 20, 2021 20:32

fixing constant capitalizations

08f55f0

remove extra debug prints and fix TSa_Family/cint issue

9046a00

remove extra debug prints and fix TSa_Family/cint issue

8f789ed

Merge branch 'add-zephyr-os' of github.com:elcritch/Nim into add-zeph…

618b8b5

…yr-os

fixing PR issues

1e1aea0

fixing constant capitalizations

26208a3

remove extra debug prints and fix TSa_Family/cint issue

65e2d40

remove extra debug prints and fix TSa_Family/cint issue

e2751e0

Remerge

a1f4bc0

fixing constant capitalizations

cee2abd

remove extra debug prints and fix TSa_Family/cint issue

2d13ca7

remove extra debug prints and fix TSa_Family/cint issue

998c753

fixing PR issues

c2c0e91

Merge branch 'add-zephyr-os' of github.com:elcritch/Nim into add-zeph…

26e2181

…yr-os

Araq reviewed Oct 21, 2021

View reviewed changes

lib/system/mm/malloc.nim Show resolved Hide resolved

elcritch commented Oct 22, 2021

View reviewed changes

lib/pure/nativesockets.nim Show resolved Hide resolved

fix maxDescriptors on zephyr/freertos

77b3ff6

Araq reviewed Oct 22, 2021

View reviewed changes

Jaremy Creechley and others added 4 commits October 22, 2021 16:27

move maxDescriptors to selector.nim -- fixes compile issue

2f6628a

change realloc impl on zephyr to match ansi c behavior

6d7030f

change realloc impl on zephyr to match ansi c behavior

212fb7e

force compileOnly mode for tlwip

e35cbf6

Araq merged commit 141b76e into nim-lang:devel Oct 24, 2021

elcritch deleted the add-zephyr-os branch October 24, 2021 09:54

Clyybber mentioned this pull request Feb 25, 2022

Merge upstream changes nim-works/nimskull#240

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Zephyr Support #19003

Add Zephyr Support #19003

elcritch commented Oct 16, 2021

elcritch Oct 16, 2021

Araq Oct 16, 2021

Araq Oct 16, 2021 •

edited

Loading

elcritch Oct 19, 2021

Araq Oct 16, 2021 •

edited

Loading

elcritch Oct 19, 2021

elcritch Oct 19, 2021

Araq commented Oct 16, 2021

elcritch commented Oct 21, 2021

Araq Oct 22, 2021

elcritch Oct 23, 2021

Araq Oct 22, 2021

elcritch Oct 22, 2021 •

edited

Loading

elcritch Oct 23, 2021

elcritch commented Oct 23, 2021

Add Zephyr Support #19003

Add Zephyr Support #19003

Conversation

elcritch commented Oct 16, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Araq Oct 16, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Araq Oct 16, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Araq commented Oct 16, 2021

elcritch commented Oct 21, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

elcritch Oct 22, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

elcritch commented Oct 23, 2021

Araq Oct 16, 2021 •

edited

Loading

Araq Oct 16, 2021 •

edited

Loading

elcritch Oct 22, 2021 •

edited

Loading