Find inconsistencies between the intel intrinsics XML file and the Rust code

[gwenn]

XSLT (data.xsl):

<?xml version="1.0"?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
    <xsl:output method="text" omit-xml-declaration="yes"/>
 
    <xsl:variable name="line">
        <xsl:text>&#10;</xsl:text>
    </xsl:variable>
    <xsl:variable name="sep">
        <xsl:text>;</xsl:text>
    </xsl:variable>
 
    <xsl:template match="/">
        <xsl:for-each select="intrinsics_list/intrinsic">
            <xsl:variable name="name" select="@name"/>
            <xsl:variable name="cpuid" select="CPUID"/>
            <xsl:variable name="instruction" select="instruction/@name"/>
            <xsl:value-of select="concat($name, $sep, $cpuid, $sep, $instruction, $line)"/>
        </xsl:for-each>
    </xsl:template>
</xsl:stylesheet>

And a script:

# Extract instrinsics from XML file
xsltproc data.xsl intel-intrinsics-3.3.15.xml > intel-intrinsics-3.3.15.csv
# Load intel intrinsics into an SQLite db
rm -f intrinsics.db
sqlite3 -batch intrinsics.db 2> /dev/null <<EOF
CREATE TABLE intrinsics(name TEXT NOT NULL UNIQUE, cpuid TEXT, instruction TEXT);
.separator ';'
.import intel-intrinsics-3.3.15.csv intrinsics
EOF

# Extract implemented intrinsics from Rust files
for cpuid in abm avx avx2 bmi bmi2 sse sse2 sse3 sse41 sse42 ssse3; do

rm -f $cpuid.csv
find coresimd/ -type f -name $cpuid.rs -print0 | xargs -0 grep -h '^#\[cfg_attr(.*assert_instr(\|pub unsafe fn' | sed -e 's/^#\[cfg_attr(.*assert_instr(\([^),]*\)[,)].*$/\1/' | sed -e 's/^pub unsafe fn \([^(]*\)(.*$/\1/' | awk '/^_/{print $0";"p; p=""}!/^_/{p=$0}' > $cpuid.csv
# Load them the SQLite db
sqlite3 -batch intrinsics.db <<EOF
CREATE TABLE $cpuid(name TEXT NOT NULL UNIQUE, instruction TEXT);
.separator ';'
.import $cpuid.csv $cpuid
-- find inconsistencies (or missing implementation)
select i.name, i.cpuid, i.instruction, s.instruction
from intrinsics i
left outer join $cpuid s on s.name = i.name
where i.cpuid like '$cpuid'
and i.instruction <> s.instruction
order by 1;
EOF

done

[gnzlbg]

Could you upload the files as a gist (it lets you specify different files with filenames, extensions, etc), and also the results?

I've tried to keep #40 updated with all the recent commits but I haven't checked that the lists maintained there are complete or accurate.

To generate the lists in #148 and #170 I used some python scripts that basically scan the reference documentation and output the intrinsics as "rust functions" (e.g. fn foo(f32x2, f32x2) -> f32x4;). I had in mind that once we implement these intrinsics we can just scan the source code like you are doing to see if all the intrinsics are implemented and the type arguments match, so thank you for doing this work!

I've been toying with the idea of doing the same for the Intel Intrinsics Guide, but while one can produce the same type of file with rust signatures, the types wouldn't tell us as much (but we would still be able to check whether we have implemented all intrinsics or not).

The cool thing about getting this set up in ci somehow (once the intrinsics are done, but maybe we could start doing it already) is that at least Intel's and ARM's online data-bases are automatically updated, so we would be warned once new intrinsics for these targets are added (new intrinsics are typically supported by the Linux kernel and GCC and LLVM before the data-bases are updated).

[gwenn]

Ok, the files are here