build Phobos with -dip1000

[WalterBright]

To illuminate remaining issues.

[dlang-bot]

Thanks for your pull request, @WalterBright!

Bugzilla references

Your PR doesn't reference any Bugzilla issue.

If your PR contains non-trivial changes, please reference a Bugzilla issue or create a manual changelog.

Testing this PR locally

If you don't have a local development environment setup, you can use Digger to test this PR:

dub fetch digger
dub run digger -- build "master + phobos#6547"

[JackStouffer]

This is one of the bigger problems is in std.getopt: #6281

There was also a forum post on this issue: https://forum.dlang.org/post/[email protected]

[wilzbach]

FWIW you can just do make -f posix.mak DFLAGS="-dip1000" to test this locally.
Also note that due to the still existing name mangling problem we recently removed the -dip1000 from the posix.mak - see #6443 and dlang/dmd#8124.

Also you could have simply looked at dip1000.mak to see what's still missing.
There's also a GitHub project for this: https://github.com/dlang/phobos/projects/6

[WalterBright]

@wilzbach those are all good initiatives, but the trouble is there is little to no progress.

[Geod24]

The last two months might have been pretty quiet, but there was definitely some work done on it

The thing is, there is a fundamental flaw in the process that was put forth for a soft -dip1000 transition: the mangling of functions differ. And it's quite a show stopper.

[WalterBright]

I know it's a show stopper. But we can start with fixing Phobos not even compiling with dip1000.

[wilzbach]

1. Phobos does compile with -dip1000 since a year or so. It's just the tests build that fails
2. Yep as @Geod24 mentioned people lost their interest in -dip1000 for many reasons (the name mangling is just one of them)
3. As mentioned I don't understand the value of this PR over the existing dip1000.mak

[WalterBright]

just the test build that fails

This looks like a release build failure:

../dmd/generated/freebsd/release/64/dmd  -conf= -I../druntime/import  -w -de -dip1000 -m64 -fPIC -transition=complex -O -release -shared -debuglib= -defaultlib= -ofgenerated/freebsd/release/64/libphobos2.so.0.80.1 -L-soname=libphobos2.so.0.80 ../druntime/generated/freebsd/release/64/libdruntime.so.a  std/array.d std/ascii.d std/base64.d std/bigint.d std/bitmanip.d std/compiler.d std/complex.d std/concurrency.d std/conv.d std/csv.d std/demangle.d std/encoding.d std/exception.d std/file.d std/format.d std/functional.d std/getopt.d std/json.d std/math.d std/mathspecial.d std/meta.d std/mmfile.d std/numeric.d std/outbuffer.d std/parallelism.d std/path.d std/process.d std/random.d std/signals.d std/socket.d std/stdint.d std/stdio.d std/string.d std/system.d std/traits.d std/typecons.d std/uni.d std/uri.d std/utf.d std/uuid.d std/variant.d std/xml.d std/zip.d std/zlib.d std/algorithm/comparison.d std/algorithm/iteration.d std/algorithm/mutation.d std/algorithm/package.d std/algorithm/searching.d std/algorithm/setops.d std/algorithm/sorting.d std/container/array.d std/container/binaryheap.d std/container/dlist.d std/container/package.d std/container/rbtree.d std/container/slist.d std/container/util.d std/datetime/date.d std/datetime/interval.d std/datetime/package.d std/datetime/stopwatch.d std/datetime/systime.d std/datetime/timezone.d std/digest/crc.d std/digest/hmac.d std/digest/md.d std/digest/murmurhash.d std/digest/package.d std/digest/ripemd.d std/digest/sha.d std/experimental/allocator/common.d std/experimental/allocator/gc_allocator.d std/experimental/allocator/mallocator.d std/experimental/allocator/mmap_allocator.d std/experimental/allocator/package.d std/experimental/allocator/showcase.d std/experimental/allocator/typed.d std/experimental/allocator/building_blocks/affix_allocator.d std/experimental/allocator/building_blocks/aligned_block_list.d std/experimental/allocator/building_blocks/allocator_list.d std/experimental/allocator/building_blocks/ascending_page_allocator.d std/experimental/allocator/building_blocks/bucketizer.d std/experimental/allocator/building_blocks/fallback_allocator.d std/experimental/allocator/building_blocks/free_list.d std/experimental/allocator/building_blocks/free_tree.d std/experimental/allocator/building_blocks/bitmapped_block.d std/experimental/allocator/building_blocks/kernighan_ritchie.d std/experimental/allocator/building_blocks/null_allocator.d std/experimental/allocator/building_blocks/package.d std/experimental/allocator/building_blocks/quantizer.d std/experimental/allocator/building_blocks/region.d std/experimental/allocator/building_blocks/scoped_allocator.d std/experimental/allocator/building_blocks/segregator.d std/experimental/allocator/building_blocks/stats_collector.d std/experimental/logger/core.d std/experimental/logger/filelogger.d std/experimental/logger/nulllogger.d std/experimental/logger/multilogger.d std/experimental/logger/package.d std/net/curl.d std/net/isemail.d std/experimental/all.d std/experimental/checkedint.d std/experimental/typecons.d std/range/interfaces.d std/range/package.d std/range/primitives.d std/regex/package.d std/regex/internal/generator.d std/regex/internal/ir.d std/regex/internal/parser.d std/regex/internal/backtracking.d std/regex/internal/tests.d std/regex/internal/tests2.d std/regex/internal/thompson.d std/regex/internal/kickstart.d etc/c/curl.d etc/c/odbc/sql.d etc/c/odbc/sqlext.d etc/c/odbc/sqltypes.d etc/c/odbc/sqlucode.d etc/c/sqlite3.d etc/c/zlib.d std/algorithm/internal.d std/digest/digest.d std/internal/cstring.d std/internal/digest/sha_SSSE3.d std/internal/math/biguintcore.d std/internal/math/biguintnoasm.d std/internal/math/biguintx86.d std/internal/math/errorfunction.d std/internal/math/gammafunction.d std/internal/scopebuffer.d std/internal/test/dummyrange.d std/internal/test/range.d std/internal/unicode_comp.d std/internal/unicode_decomp.d std/internal/unicode_grapheme.d std/internal/unicode_norm.d std/internal/unicode_tables.d std/typetuple.d generated/freebsd/release/64/etc/c/zlib/adler32.o generated/freebsd/release/64/etc/c/zlib/compress.o generated/freebsd/release/64/etc/c/zlib/crc32.o generated/freebsd/release/64/etc/c/zlib/deflate.o generated/freebsd/release/64/etc/c/zlib/gzclose.o generated/freebsd/release/64/etc/c/zlib/gzlib.o generated/freebsd/release/64/etc/c/zlib/gzread.o generated/freebsd/release/64/etc/c/zlib/gzwrite.o generated/freebsd/release/64/etc/c/zlib/infback.o generated/freebsd/release/64/etc/c/zlib/inffast.o generated/freebsd/release/64/etc/c/zlib/inflate.o generated/freebsd/release/64/etc/c/zlib/inftrees.o generated/freebsd/release/64/etc/c/zlib/trees.o generated/freebsd/release/64/etc/c/zlib/uncompr.o generated/freebsd/release/64/etc/c/zlib/zutil.o
std/experimental/logger/filelogger.d(133): Error: `@safe` function `std.experimental.logger.filelogger.FileLogger.beginLogMsg` cannot call `@system` function `std.experimental.logger.core.systimeToISOString!(LockingTextWriter).systimeToISOString`
std/experimental/logger/filelogger.d(134): Error: `@safe` function `std.experimental.logger.filelogger.FileLogger.beginLogMsg` cannot call `@system` function `std.format.formattedWrite!(LockingTextWriter, char, string, string, int).formattedWrite`
std/experimental/logger/filelogger.d(143): Error: `@safe` function `std.experimental.logger.filelogger.FileLogger.logMsgPart` cannot call `@system` function `std.format.formattedWrite!(LockingTextWriter, char, const(char)[]).formattedWrite`
std/experimental/logger/filelogger.d(143): Error: scope variable `msg` assigned to non-scope parameter `_param_2` calling std.format.formattedWrite!(LockingTextWriter, char, const(char)[]).formattedWrite
std/experimental/logger/filelogger.d(133): Error: `@safe` function `std.experimental.logger.filelogger.FileLogger.beginLogMsg` cannot call `@system` function `std.experimental.logger.core.systimeToISOString!(LockingTextWriter).systimeToISOString`
std/experimental/logger/filelogger.d(134): Error: `@safe` function `std.experimental.logger.filelogger.FileLogger.beginLogMsg` cannot call `@system` function `std.format.formattedWrite!(LockingTextWriter, char, string, string, int).formattedWrite`
std/experimental/logger/filelogger.d(143): Error: `@safe` function `std.experimental.logger.filelogger.FileLogger.logMsgPart` cannot call `@system` function `std.format.formattedWrite!(LockingTextWriter, char, const(char)[]).formattedWrite`
std/experimental/logger/filelogger.d(143): Error: scope variable `msg` assigned to non-scope parameter `_param_2` calling std.format.formattedWrite!(LockingTextWriter, char, const(char)[]).formattedWrite
gmake: *** [posix.mak:306: generated/freebsd/release/64/libphobos2.a] Error 1

[jmdavis]

We should have a build of some kind that runs with the auto-tester and circleci and all that which builds with -dip1000 so that we can catch all of the problems and ensure that Phobos builds with -dip1000 and that it continues to build with -dip1000, but actually merging this PR to make Phobos build with -dip1000 normally would be a disaster given the compatibility issues.

[wilzbach]

This looks like a release build failure:

Yep, sorry at least it used to compile (see e.g. #5915 (comment))

@jmdavis we used to have this, but purposefully disabled it because of the name mangling issues.
See e.g. #6443

Also as I'm not sure everyone understood it, dip1000.mak has been part of the build process for four months now (at the moment only CircleCi runs the tests with -dip1000).
For reference here are the twelve modules that still need fixing:

> cat dip1000.mak | grep "[-]dip25"
aa[std.file]=-dip25 # probably already fixed (std.uni); currently: undefined symbol  pure nothrow @nogc return @safe std.uni.SliceOverIndexed!(std.uni.Grapheme).SliceOverIndexed std.uni.SliceOverIndexed!(std.uni.Grapheme).SliceOverIndexed.opSlice()
aa[std.format]=-dip25 # @system function std.range.primitives.put
aa[std.outbuffer]=-dip25 # DROP: cannot call @system function std.outbuffer.OutBuffer.writef!(char, int).writef
aa[std.path]=-dip25 #    TODO
aa[std.stdio]=-dip25 #    TODO
aa[std.algorithm.searching]=-dip25 # depends on https://github.com/dlang/phobos/pull/6246 merged and std.algorithm.comparison fixed
aa[std.algorithm.sorting]=-dip25 # i.a. depends on std.algorithm.searching? and a fix for writefln
aa[std.container.rbtree]=-dip25 # DROP
aa[std.container.util]=-dip25 # depends on rbtree and slist = -dip1000
aa[std.experimental.allocator.package]=-dip25 #    Linker errors: undefined reference  pure nothrow @property @nogc return @safe std.internal.test.dummyrange.DummyRange!(0, 1, 1, uint[]).DummyRange std.internal.test.dummyrange.DummyRange!(0, 1, 1, uint[]).DummyRange.save()
aa[std.experimental.logger.filelogger]=-dip25 # merged https://github.com/dlang/phobos/pull/6266; depends on https://github.com/dlang/phobos/pull/5915 ? and a fix for: std.format.formattedWrite
aa[std.range.package]=-dip25 # reference to local variable a / b assigned to non-scope parameter _param_1 / _param_2 calling std.range.chooseAmong!(RefAccessRange, RefAccessRange).chooseAmong

[wilzbach]

As long as the name mangling issue isn't fixed, we can't merge this.

[WalterBright]

As long as the name mangling issue isn't fixed, we can't merge this.

I understand that. First, let's get it to compile.

[WalterBright]

Currently blocked by dlang/dmd#8346

[andre2007]

As dlang/dmd#8346 was merged, is this pr still blocked?

[thewilsonator]

well

std/file.d(364): Error: scope variable `name` may not be returnedstd/file.d(364): 
Error: scope variable `name` may not be returned
std/file.d(3009): Error: scope variable `pathname` may not be returned
std/file.d(3009): Error: scope variable `pathname` may not be returned
std/file.d(3037): Error: template instance `std.file.ensureDirExists!()` error instantiating
std/file.d(3037): Error: template instance `std.file.ensureDirExists!()` error instantiating
std/experimental/logger/filelogger.d(144): Error: scope variable `msg` assigned to non-scope parameter `_param_2` calling std.format.formattedWrite!(LockingTextWriter, char, const(char)[]).formattedWrite
std/experimental/logger/filelogger.d(144): Error: scope variable `msg` assigned to non-scope parameter `_param_2` calling std.format.formattedWrite!(LockingTextWriter, char, const(char)[]).formattedWrite

but not blocked by dmd.

[wilzbach]

Superseded by #6931