Conversion from Singular to Sage

[simon-king-jena]

On sage-devel, Francisco Botana complained about some shortcomings of the conversion from Singular (pexpect interface) to Sage.

I think the conversions provided by this patch are quite thorough.

First of all, the patch provides a conversion of base rings, even with minpoly, with complicated block, matrix and weighted orders (note that one needs #11316) and even quotient rings:

sage: singular.eval('ring r1 = (9,x),(a,b,c,d,e,f),(M((1,2,3,0)),wp(2,3),lp)')
'ring r1 = (9,x),(a,b,c,d,e,f),(M((1,2,3,0)),wp(2,3),lp);'
sage: R = singular('r1').sage_basering()
sage: R
Multivariate Polynomial Ring in a, b, c, d, e, f over Finite Field in x of size 3^2
sage: R.term_order()
Block term order with blocks:
(Matrix term order with matrix
[1 2]
[3 0],
 Weighted degree reverse lexicographic term order with weights (2, 3),
 Lexicographic term order of length 2)

sage: singular.eval('ring r3 = (3,z),(a,b,c),dp')
'ring r3 = (3,z),(a,b,c),dp;'
sage: singular.eval('minpoly = 1+z+z2+z3+z4')
'minpoly = 1+z+z2+z3+z4;'
sage: singular('r3').sage_basering()
Multivariate Polynomial Ring in a, b, c over Univariate Quotient Polynomial Ring in z over Finite Field of size 3 with modulus z^4 + z^3 + z^2 + z + 1

sage: singular.eval('ring r5 = (9,a), (x,y,z),lp') 
'ring r5 = (9,a), (x,y,z),lp;'
sage: Q = singular('std(ideal(x^2,x+y^2+z^3))', type='qring')
sage: Q.sage_basering()
Quotient of Multivariate Polynomial Ring in x, y, z over Finite Field in a of size 3^2 by the ideal (y^4 - y^2*z^3 + z^6, x + y^2 + z^3)

By consequence, it is now straight forward to convert polynomials or ideals to sage:

sage: singular.eval('ring R = integer, (x,y,z),lp') 
'// ** You are using coefficient rings which are not fields...'
sage: I = singular.ideal(['x^2','y*z','z+x']) 
sage: I.sage()  # indirect doctest
Ideal (x^2, y*z, x + z) of Multivariate Polynomial Ring in x, y, z over Integer Ring

# Note that conversion of a Singular string to a Sage string was missing
sage: singular('ringlist(basering)').sage()
[['integer'], ['x', 'y', 'z'], [['lp', (1, 1, 1)], ['C', (0)]], Ideal (0) of Multivariate Polynomial Ring in x, y, z over Integer Ring]

sage: singular.eval('ring r10 = (9,a), (x,y,z),lp')
'ring r10 = (9,a), (x,y,z),lp;'
sage: Q = singular('std(ideal(x^2,x+y^2+z^3))', type='qring')
sage: Q.sage()
Quotient of Multivariate Polynomial Ring in x, y, z over Finite Field in a of size 3^2 by the ideal (y^4 - y^2*z^3 + z^6, x + y^2 + z^3)
sage: singular('x^2+y').sage()
x^2 + y
sage: singular('x^2+y').sage().parent()
Quotient of Multivariate Polynomial Ring in x, y, z over Finite Field in a of size 3^2 by the ideal (y^4 - y^2*z^3 + z^6, x + y^2 + z^3)

---

Apply

1. attachment: trac11431_singular_sage_conversion.patch
2. attachment: trac11431_singular_sage_documentation.patch
   to the Sage library.

Depends on #11316
Depends on #11645

Upstream: None of the above - read trac for reasoning.

CC: @malb

Component: interfaces

Author: Simon King

Reviewer: Martin Albrecht

Merged: sage-4.7.2.alpha3

Issue created by migration from https://trac.sagemath.org/ticket/11431

[simon-king-jena]

comment:1

Note to the reviewer: I did not check whether the doc strings look nice in html.

[simon-king-jena]

Conversion Singular -> Sage

[simon-king-jena]

comment:2

Attachment: trac11431_singular_sage_conversion.patch.gz

So far, the method sage() has not been mentioned at all in the documentation of sage.interfaces.singular! Therefore I added a second patch, providing some use cases of .sage() in the short tutorial of sage.interfaces.singular.

The examples that are visible in the html version of the documentation still do not cover all features (quotient rings etc.), but I guess if a user knows about the existence of sage() (s)he will guess its correct usage.

[malb]

comment:3

• can we rename sage_basering to sage_global_ring. In Sage base_ring has a special meaning which is not meant in your patch. I think global ring captures it.
  • don't real fields also have a precision in Singular?
  • Previously, we enforced and assumed not short, but it's good you're taking it into account.
  • Can you also fix "If it is provided, then you must take care OFF it."?
  • shouldn't TermOrder_from_Singular be lower-case since it's a function? Or were you aiming for upper-case because it's a constructor?
  • In "Invalid termorder in Singular" shouldn't it be term order, i.e. with a space?

[malb]

Reviewer: Martin Albrecht

[simon-king-jena]

comment:4

Hi Martin!

Replying to @malb:

• can we rename sage_basering to sage_global_ring. In Sage base_ring has a special meaning which is not meant in your patch. I think global ring captures it.

Sounds fine to me. Obviously I took the name sage_basering from Singular, but you are right that it might be confusing for people who know the notion base ring only from Sage.

• don't real fields also have a precision in Singular?

Right, that should be taken care of as well.

• Can you also fix "If it is provided, then you must take care OFF it."?

Why "off"? With two f?

Wouldn't it be correct English to say "If it is provided, then you must take care that it matches the current ring in Singular"? (i.e., I forgot the "that" in my patch)

• shouldn't TermOrder_from_Singular be lower-case since it's a function? Or were you aiming for upper-case because it's a constructor?

I made it upper case because it returns a TermOrder (that is how the class is called). But actually I was ignorant to the convention that functions are lower-case. So I'll change it.

• In "Invalid termorder in Singular" shouldn't it be term order, i.e. with a space?

Yep!

[simon-king-jena]

comment:5

Replying to @simon-king-jena:

...

• Can you also fix "If it is provided, then you must take care OFF it."?

Why "off"? With two f?

Wouldn't it be correct English to say "If it is provided, then you must take care that it matches the current ring in Singular"? (i.e., I forgot the "that" in my patch)

Perhaps not. But "If it is provided, then you have to make sure that it matches..." is fine, isn't it?

• shouldn't TermOrder_from_Singular be lower-case since it's a function? Or were you aiming for upper-case because it's a constructor?

I made it upper case because it returns a TermOrder (that is how the class is called). But actually I was ignorant to the convention that functions are lower-case. So I'll change it.

• In "Invalid termorder in Singular" shouldn't it be term order, i.e. with a space?

Yep!

[simon-king-jena]

Show examples of .sage() to the singular interface tutorial. Fix some details of the previous patch.

[simon-king-jena]

comment:6

Attachment: trac11431_singular_sage_documentation.patch.gz

I've put my changes into the second patch, so, please replace that when you test it.

Names: I changed sage_basering into sage_global_ring and TermOrder_from_Singular into termorder_from_singular.

Precision: Singular does not tell the precision of a real field in charstr(), it does so only for complex fields. But the precision both of real and complex fields is part of ringlist(). So, both in the real and the complex case, I take the precision from there, not from charstr().

Grammar: It is now "Invalid term order" (not "termorder") and "If it is provided, then you have to make sure that it matches..." (not "If it is provided, then you must take care it matches...")

Tests: Of course I added a test showing that the precision of real fields is taken care of. And it turned out that one doctest needed a fix, because when I created it I used a different term order (so that the same polynomial was printed in a different way).

[malb]

comment:7

I'll give this a positive review: I agree with Simon's fixes (and the patchbot can take care of any doctest failures etc.)

[simon-king-jena]

comment:8

Thank you! And the patchbot actually says that the doctests pass...

[jdemeyer]

Merged: sage-4.7.1.alpha3

[jdemeyer]

Changed merged from sage-4.7.1.alpha3 to none

[jdemeyer]

comment:10

There are some issues on Solaris. On the "marks" and "t2" buildbots, I get the following error (identical on both systems):

sage -t -long  -force_lib devel/sage/sage/rings/polynomial/term_order.py
**********************************************************************
File "/scratch/buildbot/sage/t2-1/t2_full/build/sage-4.7.1.alpha3/devel/sage-main/sage/rings/polynomial/term_order.py", line 1878:
    sage: singular.eval('ring r1 = (9,x),(a,b,c,d,e,f),(M((1,2,3,0)),wp(2,3),lp)')
Exception raised:
    Traceback (most recent call last):
      File "/scratch/buildbot/sage/t2-1/t2_full/build/sage-4.7.1.alpha3/local/bin/ncadoctest.py", line 1231, in run_one_test
        self.run_one_example(test, example, filename, compileflags)
      File "/scratch/buildbot/sage/t2-1/t2_full/build/sage-4.7.1.alpha3/local/bin/sagedoctest.py", line 38, in run_one_example
        OrigDocTestRunner.run_one_example(self, test, example, filename, compileflags)
      File "/scratch/buildbot/sage/t2-1/t2_full/build/sage-4.7.1.alpha3/local/bin/ncadoctest.py", line 1172, in run_one_example
        compileflags, 1) in test.globs
      File "<doctest __main__.example_64[2]>", line 1, in <module>
        singular.eval('ring r1 = (9,x),(a,b,c,d,e,f),(M((1,2,3,0)),wp(2,3),lp)')###line 1878:
    sage: singular.eval('ring r1 = (9,x),(a,b,c,d,e,f),(M((1,2,3,0)),wp(2,3),lp)')
      File "/scratch/buildbot/sage/t2-1/t2_full/build/sage-4.7.1.alpha3/local/lib/python/site-packages/sage/interfaces/singular.py", line 567, in eval
        raise RuntimeError, 'Singular error:\n%s'%s
    RuntimeError: Singular error:
       ? cannot open `gftables/9`
       ? cannot make ring
       ? error occurred in or before STDIN line 33: `ring r1 = (9,x),(a,b,c,d,e,f),(M((1,2,3,0)),wp(2,3),lp);`
       ? expected ring-expression. type 'help ring;'
**********************************************************************
File "/scratch/buildbot/sage/t2-1/t2_full/build/sage-4.7.1.alpha3/devel/sage-main/sage/rings/polynomial/term_order.py", line 1881:
    sage: termorder_from_singular(singular)
Expected:
    Block term order with blocks:
    (Matrix term order with matrix
    [1 2]
    [3 0],
     Weighted degree reverse lexicographic term order with weights (2, 3),
     Lexicographic term order of length 2)
Got:
    Block term order with blocks:
    (Lexicographic term order of length 3,
     Degree lexicographic term order of length 5,
     Lexicographic term order of length 2)
**********************************************************************

[simon-king-jena]

comment:11

Hi Jeroen,

Replying to @jdemeyer:

sage: singular.eval('ring r1 = (9,x),(a,b,c,d,e,f),(M((1,2,3,0)),wp(2,3),lp)')

...
RuntimeError: Singular error:
? cannot open gftables/9
? cannot make ring
? error occurred in or before STDIN line 33: ring r1 = (9,x),(a,b,c,d,e,f),(M((1,2,3,0)),wp(2,3),lp);
? expected ring-expression. type 'help ring;'

---

That is very strange. I can not imagine that my patch introduced that bug: It changes the conversion from Singular to Sage, but it should not change Singular itself (and what you get in your example is an error from Singular, not from the interface).

Hence, I wonder whether my patch has introduced or uncovered that bug.

Can you test on marks and t2 with unpatched Sage whether the line

sage: singular.eval('ring r1 = (9,x),(a,b,c,d,e,f),(M((1,2,3,0)),wp(2,3),lp)')

fails?

[simon-king-jena]

Work Issues: Singular trouble on Solaris

[simon-king-jena]

comment:16

As I have indicated on #11645, there is a potential work around for the missing gftables: If one has a polynomial ring over a finite non-prime field in Sage and converts it into Singular, then the missing gftable is created. Hence, one could ensure that the problematic doc test passes by repending that test with another one that converts a ring from Sage into Singular.

That would be a nasty work-around, but at least it would mean that my patch could be used.

[simon-king-jena]

Work around a bug on Solaris revealed by the first patch

[simon-king-jena]

Changed upstream from Reported upstream. Little or no feedback. to None of the above - read trac for reasoning.

[simon-king-jena]

Changed work issues from Singular trouble on Solaris to none

[simon-king-jena]

comment:17

Attachment: trac11431_solaris_workaround.patch.gz

I provided a third patch that uses - as a workaround - the fact that libsingular is able to create missing gftables files.

On Singular trac, Hannes said "No, gftables are parts of the sources and will be copied during installation, but usually not computed (the program for that, a part of factory, will not be build during a standard compilation and installation).

So, I wonder why the files aren't copied on Solaris and how libsingular is able to create them. But I think the third patch (that isn't tested, yet) should suffice to make the doc tests of the first two patches work on solaris.

Back to "needs review" - I hope that someone can doctest on Solaris.

[simon-king-jena]

comment:18

Note that #11645 also fixes the problem with the new doc test.

So, if someone would like to review my patches, I see two ways to make things work:

Either

1. get Singular does not find "gftables/9" on Solaris #11645 and attachment: trac11431_singular_sage_conversion.patch and attachment: trac11431_singular_sage_documentation.patch

or

2. get attachment: trac11431_singular_sage_conversion.patch, attachment: trac11431_singular_sage_documentation.patch and attachment: trac11431_solaris_workaround.patch.

[jdemeyer]

Changed dependencies from #11316 to #11316, #11645

[jdemeyer]

comment:19

What's the status of this ticket now? Since #11645 has been merged in sage-4.7.1, we can assume that as prerequisite. Which patches should be applied then?

If somebody can give these patches a "tentative positive review" (without testing on Solaris), I can test them on the buildbot machines, including Solaris.

[jdemeyer]

comment:20

Replying to @jdemeyer:

If somebody can give these patches a "tentative positive review" (without testing on Solaris), I can test them on the buildbot machines, including Solaris.

To the reviewer: if you do give such a "tentative positive review", you can set the status to "positive review" (even if it has not been tested properly).

[malb]

comment:21

Simon, can you rebase your patches to 4.7.1?

[simon-king-jena]

comment:22

Replying to @malb:

Simon, can you rebase your patches to 4.7.1?

Since #11645 got fixed, I have only applied the first two patches, to sage-4.7.2.alpha1. They applied without fuzz. Why is it needed to rebase them?

[malb]

comment:23

Hi, I get this:

SAGE_ROOT=/mnt/usb1/scratch/malb/sage-4.7.2.alpha0
(sage subshell) sage:sage malb$ hg qimport https://github.com/sagemath/sage-prod/files/10653045/trac11431_singular_sage_conversion.patch.gz && hg qpush
adding trac11431_singular_sage_conversion.patch to series file
applying trac11431_singular_sage_conversion.patch
patching file sage/rings/polynomial/term_order.py
Hunk #1 FAILED at 1857
1 out of 1 hunks FAILED -- saving rejects to file sage/rings/polynomial/term_order.py.rej
patch failed, unable to continue (try -v)
patch failed, rejects left in working dir
errors during apply, please fix and refresh trac11431_singular_sage_conversion.patch
SAGE_ROOT=/mnt/usb1/scratch/malb/sage-4.7.2.alpha0

[simon-king-jena]

comment:24

Hi Martin,

you forgot the other dependency: #11316 was only merged in sage-4.7.2.alpha1, but you started with alpha0.

[malb]

comment:25

Right, now it applies!

[malb]

comment:26

applies & doctests pass. I read the patch a while ago so positive review it is.

[nexttime]

comment:27

Simon, are you ok with dropping the (IMHO obsolete) Solaris patch?

[simon-king-jena]

comment:28

Replying to @nexttime:

Simon, are you ok with dropping the (IMHO obsolete) Solaris patch?

Yes, it can be dropped, since that problem has been successfully dealt with on a different ticket.

[nexttime]

Changed dependencies from #11316, #11645 to #11316 #11645

[nexttime]

Merged: sage-4.7.2.alpha3