GAP-Python communication hangs in both directions
Closed this issue · 32 comments
c = SCSCPCLI('localhost') hangs forever.
The same happens when GAP client connects to the Python server. Server displays
INFO:demo_server:New connection from 127.0.0.1:59002
DEBUG:demo_server.127.0.0.1:Sending PI: <?scscp scscp_versions="1.3" ?>
and then nothing happens.
I will try to look later, but I think you may need to put \n at the end of each processing instruction and message.
Are you sure line feeds are part of the XML PI specification? I think GAP is doing the wrong thing if it expects \n.
Yes :) Adding it will certainly not breach SCSCP rules and will please GAP. We did it before with other counterparts.
I'm perfectly ok sending \n to help compatibility with other implementations. However I strongly encourage said implementations to fix this :)
Do you have a box running gap-scscsp where I can ssh and do some tests? I still haven't managed to have a local install of it.
Yes, demo-server at chrystal.mcs.st-andrews.ac.uk at default SCSCP port - please try! (you can try it remotely, you don't have to ssh into it).
Well, adding the \n makes the GAP server more talkative, however it kicks me out when I try to fetch the heads. I'll have to investigate this further.
@defeo can you give me a branch where \n are, so I could try it?
Pushed a commit with more line feeds, and more verbose debugging. Run with
>>> import scscp, logging
>>> logging.basicConfig(level=10)
>>> c = scscp.SCSCPCLI('localhost')
to see debugging messages.
However, the GAP server keeps kicking me out when I send this OM message:
<om:OMOBJ xmlns:om="http://www.openmath.org/OpenMath" version="2.0"><om:OMATTR><om:OMATP><om:OMS name="call_id" cd="scscp1"/><om:OMSTR>18a8b774-f396-11e6-b287-101f7477d447</om:OMSTR><om:OMS name="option_return_object" cd="scscp1"/><om:OMSTR>True</om:OMSTR></om:OMATP><om:OMA><om:OMS name="procedure_call" cd="scscp1"/><om:OMA><om:OMS name="get_allowed_heads" cd="scscp2"/></om:OMA></om:OMA></om:OMATTR></om:OMOBJ>
Alex, what version of OpenMath does GAP support? This package only supports OM 2.0. Quoting the OpenMath specs
An encoded OpenMath object is placed inside an OMOBJ element. This element can contain the elements (and integers) described above. It can take an optional version (XML) attribute which indicates to which version of the OpenMath standard it conforms. In previous versions of this standard this attribute did not exist, so any OpenMath object without such an attribute must conform to version 1 (or equivalently 1.1) of the OpenMath standard. Objects which conform to the description given in this document should have version="2.0".
The server you set up does not send the version attribute.
Ok, found the problem: GAP-scscp does not support namespaced XML. <om:OMOBJ> makes it close the connection, <OMOBJ> works fine.
GAP should:
- Give a proper SCSCP error instead of closing the connection when it is unable to parse the OM message.
- Implement namespaced XML.
I'll patch this package to allow sending unnamespaced XML in the meantime. The problem in my previous message stands still: GAP must send the version attribute if it speaks OM 2.0, otherwise no communication is possible.
All right, I've made namespace prefixes configurable in https://github.com/OpenMath/py-openmath, and have pushed to master the linefeed changes to this project.
Now I can successfully talk to your server, but the SCSCPCLI class intentionally refuses to continue the conversation because of the version issue mentioned above.
I haven't made a new pip release yet. I'll let you play around a bit more (install both packages from github), and publish when things have stabilized.
@defeo thanks. Could you please try scscp.gap-system.org again, I've restarted it and added version.
Yay!
>>> from scscp import SCSCPCLI
>>> c = SCSCPCLI('scscp.gap-system.org')
>>> c.heads.scscp_transient_1.WS_Factorial([10])
3628800
It wouldn't hurt to add an xmlns attribute to OMOBJ, I had to patch https://github.com/OpenMath/py-openmath to be less fussy about it missing.
Ok, I can add xmlns="http://www.openmath.org/OpenMath". But isn't it the case that it's optional?
It is possibly optional, I haven't checked the SCSCP specs, but, adding it doesn't hurt, and allows to verbatim copy an xml snippet from scscp into a different xml document.
Anyway, I removed the check from py-openmath, because it makes sense that it would be optional in SCSCP.
Ok, I can add xmlns="http://www.openmath.org/OpenMath". But isn't it the case that it's optional?
Actually, putting OpenMath XML into the right namespace is not optional in OpenMath, and as a consequence it should not be in SCSCP.
But of course the python implementation can be lenient on this.
if you can remove the requirement for namespaces it will be nice. If it's optional, I don't want to increase the verbosity without a reason here and there, it will increase the traffic and eventually lead to performance penalties.
With version it seems that you're right that is must be there, as the standard says - althoughI'd prefer to ignore it (maybe make it optional and omit in GAP-to-GAP cases).
Regarding "Implement namespaced XML" - we use XML parser from GAPDoc package, which IIRC is not capable of doing this at the moment. Again, is this really a part of OpenMath standard?
Regarding "Give a proper SCSCP error instead of closing the connection when it is unable to parse the OM message" - I agree. I think most of calls on the server are done via "call and catch" to return an error to the client. I've created a reminder for myself at gap-packages/scscp#7.
There are precedents of context overruling the requirement that xmlns be part of a document. For example, xmlns is required in svg images, but it is optional inside the <svg> tag of html documents. See http://stackoverflow.com/a/18468348/2569487.
However, the html specs are a mess, and there are historical reasons for this kind of leniency. In the context of SCSCP+OpenMath, I would not play with fire, and would conform to strict XML compliance. Of course, that's up to the standardization committee.
And I really don't think that the additional 40 bytes would have any impact on performance. On a network, latency would be much more important, and on unix pipes, 40B really make no difference.
As for supporting namespace prefixes, that's part of the XML specs. There are tons of xml parsers that support that, GAP would be much more compatible if it used one of those.
Anyway, these are only suggestions.
I agree with Luca here, the OpenMath should be in the right namespace.
Yes, 40 extra bytes per transation is a (tiny) performance penalty, but the real penalty has already been payed when we XML was adopted for communication. BTW for good reasons, of which namespaces are one.
It may all add up if you e.g. run parallel distributed computation with master-worker skeleton. But in that case you should probably switch to binary encoding and will have neither version nor xmlns - so ok to add it to XML.
I agree, the binary encoding is exactly for that.
BTW, does GAP support this?
Here you are:
gap> OMString(42);
"<OMOBJ xmlns=\"http://www.openmath.org/OpenMath\" version=\"2.0\"> <OMI>42</O\
MI> </OMOBJ>"
Fixing tests and making new release shortly.
@kohlhase yes, GAP support binary OpenMath: some details in http://gap-packages.github.io/openmath/doc/chap2.html (look for the word "binary")
thanks, that is good to know (I have the feeling I asked you before,... )
Then we should probably also support binary in MMT and Python.
eventually
Released new OpenMath and SCSCP
Now I can do:
>>> from scscp import SCSCPCLI
>>> c = SCSCPCLI('scscp.gap-system.org')
>>> c
<scscp.cli.SCSCPCLI object at 0x105933490>
>>> c.heads
{'scscp_transient_1': ['SCSCPStartTracing', 'Addition', 'IO_UnpickleStringAndPickleItBack',
'NrConjugacyClasses', 'ConwayPolynomial', 'SmallGroup', 'GroupIdentification',
'AutomorphismGroup', 'IdGroup512ByCode', 'Phi', 'Factorial', 'GnuExplained', 'MathieuGroup',
'TransitiveGroup', 'PrimitiveGroup', 'Multiplication', 'NextUnknownGnu', 'Identity', 'IsPrimeInt', 'Gnu',
'Determinant', 'LatticeSubgroups', 'Length', 'MatrixMultiplication', 'SCSCPStopTracing',
'AlternatingGroup', 'SymmetricGroup', 'IdGroup', 'SylowSubgroup', 'GnuWishlist', 'Size']}
>>> c.heads.scscp_transient_1.Identity([1])
1
>>> c.heads.scscp_transient_1.Determinant([ [ [ 1,2 ],[ 3,4] ] ])
-2
Number of groups of order 10000:
>>> c.heads.scscp_transient_1.Gnu([10000])
4728
What is the catalogue number of the group generated by matrices a and b?
>>> a = [ [ 0,-1], [1,-1] ]
>>> b = [ [-1,1], [0,1 ] ]
>>> c.heads.scscp_transient_1.GroupIdentification([[a,b]])
[6, 1]
Excellent, I'll push the new versions of the Python packages on PyPI tomorrow, then!
PyPI packages updated