Re: [GiNaC-list] Python wrapper for a GiNaC derived library
Ondrej Certik ondrej at certik.cz write on Thu Jul 20 00:13:29 CEST 2006
We also created swiginac (http://swiginac.berlios.de/), for those, who are interested in using ginac from python.
just my opinion: boost-python took ages to compile on my machine and the errors were completely unreadable. swig compiles much faster, and also it seems easier to learn. Do others have similar experience, or totally different?
I agree that both BoostPython and pyGiNaC have some issues and may be willing to make my library wrap in swiginac as well. However the swiginac is not supporting the new tinfo system and I cannot see right now how to patch it. Best wishes, Vladimir -- Vladimir V. Kisil email: kisilv@maths.leeds.ac.uk -- www: http://maths.leeds.ac.uk/~kisilv/
Which reminds me, do you find ginac quite difficult to extend? I find the whole ginac unnecessary complex: for example declaring new symbolic functions requires using 2 macros, adding new classes is even more complicated. Then there are several types of functions "pow" (for example), one for ex, another for numeric and another for general typemaps T1 and T2. then there are both methods of classes and standalone functions, which do the same thing. there are several almost the same constructors for most of the classes. There is the "ex" class - it seem to me it's not needed at all. In the source, it's written, its for counting references and acts like a proxy. But imho it's just making everything complicated. I am just curious, if there is some basis for those decisions, or if you just decided to do it that way. I understand that ginac just works and is doing the job (and my own code does nothing yet), so from this point of view my complains are irrelevant. But I think there were more people than me, who tried to extend ginac, and find it too much difficult, so they gave up. I like the idea of ginac and I think it would be even better to use python as the language for calling all the ginac commands (because python is exactly a language for CAS imho - the code looks much nicer and is much shorter than in C++). There is pyginac (with the problems I mentioned in my last email) and I wrote (together with Ola Skavhaug) swiginac. It's doing the job, but it's difficult to extend, as I have just said. (BTW, both pyginac and swiginac are unwrapping "ex"s, which really makes one ask the question of the necessity of ex.) Half a year ago I wrote some proof of concept of a CAS (computer algebra system) in python: http://ondrej.certik.cz/cas.php which shows, that it should be possible to rewrite ginac in much simpler way. The code is still a little bit messy, but it's really short taking into account what it can achieve already. The idea is to test everything in python first, and then maybe rewrite it to C++ for speed. The biggest lesson I learned is, that it is not that difficult to write new CAS. I wanted to write to this list later, when I have more functioning code, but because I don't have much time to continue on it, I decided to write now. Is anybody having similar experience or do you think it's not worthy to write another CAS from scratch? Of course I understand nobody has time for new venture and ginac works pretty well after all. My question is just if you think there is a room for such a thing. If not me, maybe somebody else would be encouraged to do it. I think there are people, who would implement for example the factorization (you can copy it from http://eigenmath.net/ for example) and other stuff like limits and integration, if the code weren't so complex (or am I wrong?). Ondrej P.S. I also wrote almost the same question to comp.lang.python, with no big response: http://groups.google.com/group/comp.lang.python/msg/8aa724d2642ec890?hl=en& On 7/20/06, Vladimir Kisil <kisilv@maths.leeds.ac.uk> wrote:
Ondrej Certik ondrej at certik.cz write on Thu Jul 20 00:13:29 CEST 2006
We also created swiginac (http://swiginac.berlios.de/), for those, who are interested in using ginac from python.
just my opinion: boost-python took ages to compile on my machine and the errors were completely unreadable. swig compiles much faster, and also it seems easier to learn. Do others have similar experience, or totally different?
I agree that both BoostPython and pyGiNaC have some issues and may be willing to make my library wrap in swiginac as well. However the swiginac is not supporting the new tinfo system and I cannot see right now how to patch it.
Best wishes, Vladimir -- Vladimir V. Kisil email: kisilv@maths.leeds.ac.uk -- www: http://maths.leeds.ac.uk/~kisilv/ _______________________________________________ GiNaC-list mailing list GiNaC-list@ginac.de https://www.cebix.net/mailman/listinfo/ginac-list
Ondrej Certik wrote:
I find the whole ginac unnecessary complex: for example declaring new symbolic functions requires using 2 macros, adding new classes is even more complicated. Then there are several types of functions "pow" (for example), one for ex, another for numeric and another for general typemaps T1 and T2.
The ones for ex are clear. The ones for numeric are for speed, not just for the call itself but also when it's put into another function where the numeric return type may yield another speedup. But that is not entirely undisputable and we've discussed this before, without clear outcome. The ones for general types T are for disambiguation purposes.
There is the "ex" class - it seem to me it's not needed at all. In the source, it's written, its for counting references and acts like a proxy. But imho it's just making everything complicated.
Allocating all objects on the free store and having to delete them manually (and maybe even managing refcounts manually) wouldn't make everything simpler, I suppose. Class ex came into existence since there are so many operations where we know very little about the general result type. Consequently, we cannot juggle the objects themselves on the stack, but have to deal with basic pointers. A std::vector of objects of various types (as in a general, unexpanded polynomial) would really have to be a vector of such pointers, too. Such raw pointers are better wrapped. There is passing mention to this in the FAQ <http://www.ginac.de/FAQ.html#evaluation>. Cheers -richy. -- Richard B. Kreckel <http://www.ginac.de/~kreckel/>
The ones for ex are clear. The ones for numeric are for speed, not just for the call itself but also when it's put into another function where the numeric return type may yield another speedup. But that is not entirely undisputable and we've discussed this before, without clear outcome. The ones for general types T are for disambiguation purposes.
Another option is to have just the "pow" class and construct everything just using it's constructor. I find it much less complex than having 1 class plus 3 functions. But this issue is not that important.
Allocating all objects on the free store and having to delete them manually (and maybe even managing refcounts manually) wouldn't make everything simpler, I suppose. Class ex came into existence since there are so many operations where we know very little about the general result type. Consequently, we cannot juggle the objects themselves on the stack, but have to deal with basic pointers. A std::vector of objects of various types (as in a general, unexpanded polynomial) would really have to be a vector of such pointers, too. Such raw pointers are better wrapped. There is passing mention to this in the FAQ <http://www.ginac.de/FAQ.html#evaluation>.
It seems to me that easiest way is to have one parent class ("basic") and derive everything from it (which ginac does in fact). And you return just a "basic". Wheter it is in fact "add", or "mul", or something else, will be determined at runtime. i.e. basic e=(a+b)^2 is an instance of "pow". if you expand it, one would call e.expand(). and you don't need any non class functions like expand. it will be just implemented in "pow", because only pow should know, how to expand powers. because you may want to call expand() of the basic class without retyping it to pow, it can be also mentioned in "basic" class declaration as an empty method (basically all the methods of ex, like expand or series can be implemented in basic). Cheers, Ondrej
Dear Ondrej and others, On Mon, 31 Jul 2006, Ondrej Certik wrote:
It seems to me that easiest way is to have one parent class ("basic") and derive everything from it (which ginac does in fact). And you return just a "basic". Wheter it is in fact "add", or "mul", or something else, will be determined at runtime. i.e. basic e=(a+b)^2
Yes, that would be easiest. However, I'm a bit afraid that it will not really work in C++. A power is a thing that has a basis and and exponent. So, I suppose these would need to be stored in basic. If power(a+b, 2) actually returns an object of type "power" it would get stripped of its non-basic members by the construction that you give. I sometimes wonder if it wouldn't be better if functions (not constructors) were used to construct our objects. Then there would be a function ex power(basis, exponent) and a class that would have to be called differently, say power_class. The function could dynamically allocate the object of type power_class. With this idea it would no longer be necessary to duplicate objects from stack to heap as is done now. This is not simpler, but I suppose it is more efficient. Maybe some cleverly-chosen macros could make it easier. Well, maybe it is simpler because the user will only see less objects derived from basic and only exes instead. Best wishes, Chris
Yes, that would be easiest. However, I'm a bit afraid that it will not really work in C++. A power is a thing that has a basis and and exponent. So, I suppose these would need to be stored in basic. If power(a+b, 2) actually returns an object of type "power" it would get stripped of its non-basic members by the construction that you give.
I think it wouldn't. power(a+b,2) would return an instance of a class power. power is however a subclass of basic. so in all the other methods and functions, you would accept "basic"s only. Let's say: series(basic &e, basic&x). if you need to access the power specific methods, you need to retype the basic &e, to a power (if it is a power of course). I used this concept in my cas in python (in python you actually don't have to retype the classes at all, you just call the method): http://code.google.com/p/sympy/ so I think it is the easiest and the simplest way how to do that. I am not sure however, if this is the fastest.
I sometimes wonder if it wouldn't be better if functions (not constructors) were used to construct our objects. Then there would be a function ex power(basis, exponent) and a class that would have to be called differently, say power_class. The function could dynamically allocate the object of type power_class. With this idea it would no longer be necessary to duplicate objects from stack to heap as is done now. This is not simpler, but I suppose it is more efficient. Maybe some cleverly-chosen macros could make it easier. Well, maybe it is simpler because the user will only see less objects derived from basic and only exes instead.
I don't know - I find it complicated. I am using just constructors of the particular classes to construct objects. Like this: mul(add(a,pow(b,c)),d) gives (a+b^c)*d where mul,add,pow are classes. plus I of course overloaded the operators *,/,+,- in the class basic, so you can actually write: (a+b**c)*d, but this is just syntactic sugar for mul(add(a,pow(b,c)),d). so I don't need any extra functions, any extra macros, nothing. At least to me, this looks simple. When you created ginac, at around 2001, how much time did you spend coding, to achieve the basic functionality? Couple of weeks? Or months/years? And how many people - 4? Ondrej
Ondrej Certik wrote:
When you created ginac, at around 2001, how much time did you spend coding, to achieve the basic functionality? Couple of weeks? Or months/years? And how many people - 4?
Several months, I suppose. But the very basic functionality that you seem to be discussing was mostly done by Alex Frink alone, at that time. The state was a buggy expand(), no normal(), no indices, no matrices, and a quite useless numeric class. But otherwise, juggling symbols worked fine. I have clear recollection of Jos Vermaseren asking me (just before the release of 0.7.0) how much effort had been put into the code and, then, my answer was three man-years and that was only a slight overestimate. OTOH, at that time we really had no code to look into and learn from. Cheers -richy. -- Richard B. Kreckel <http://www.ginac.de/~kreckel/>
Dear Ondrej, On Thu, 3 Aug 2006, Ondrej Certik wrote:
series(basic &e, basic&x)
Yes, as long as you are using references you are fine (... well not really, see below ...). You can call virtual methods through references and you can cast them back to references to the appropriate child-class of basic. However, at some point you will need to create real objects. This is because writing basic &e = power(a+b, 2) is not allowed. The compiler will reject it. It would create a reference out of a temporary which you cannot do in C++. Oh well, I said that you will need to create real objects, so let us write: basic e = power(a+b, 2) No! Also not good! It will create a new basic object with only the basic-parts of the power. Not the base and exponent. The problem that he compiler rejects references to temporaries is going to hunt you. The signature of the power constructor cannot be power::power(basic &b, basic &e) because you cannot pass a+b to the first (reference!) argument of the constructor. What shall we do then? Leave out the &'s. No! this will discard all the add-specific data of a+b and only copy the, ehhm well, bare basics. The point is that your Python data types are already doing for you automatically what GiNaCs exes are doing in C++: being a reference to a garbage-collected data type. I don't think it is possible to do without that.
When you created ginac, at around 2001, how much time did you spend coding, to achieve the basic functionality? Couple of weeks? Or months/years? And how many people - 4?
I didn't spend any time on it when GiNaC was created. I'm a second-generation developer. As far as I know, GiNaC was created around 1998. My first patch went into version 1.0.9 and that was in 2002. Best wishes, Chris
Yes, as long as you are using references you are fine (... well not really, see below ...). You can call virtual methods through references
that's what I mean - use references, I think I am fine, see below.
and you can cast them back to references to the appropriate child-class of basic. However, at some point you will need to create real objects. This is because writing basic &e = power(a+b, 2) is not allowed. The compiler will reject it. It would create a reference out of a temporary which you cannot do in C++. Oh well, I said that you will need to create real objects, so let us write: basic e = power(a+b, 2) No! Also not good! It will create a new basic object with only the basic-parts of the power. Not the base and exponent.
The problem that he compiler rejects references to temporaries is going to hunt you. The signature of the power constructor cannot be power::power(basic &b, basic &e) because you cannot pass a+b to the first (reference!) argument of the constructor. What shall we do then? Leave out the &'s. No! this will discard all the add-specific data of a+b and only copy the, ehhm well, bare basics.
The point is that your Python data types are already doing for you automatically what GiNaCs exes are doing in C++: being a reference to a garbage-collected data type. I don't think it is possible to do without that.
I am not sure I understand. Why cannot we use it like in the following example? I compiled it and it works as expected: #include <iostream> class basic { public: basic() {}; }; class power:public basic { public: power(basic *a, basic *b) { this->a=a; this->b=b; }; void print() { std::cout<<this->a << " ^ " << this->b << std::endl; } private: //a ... base, b... exponent basic *a,*b; }; basic* pow(basic*a, basic*b) { return new power(a,b); } int main() { basic *e1=new basic(); //some complicated expression basic *e2=new basic(); //some complicated expression basic *p=new power(e1,e2); //p->print(); //this line won't compile in C++, in python its ok ((power *)p)->print(); //OK, prints: 0x804a008 ^ 0x804a018 //because there is no operator for power in C++ to be overloaded, we can create a //function to do that basic *q=pow(e2,e1); ((power *)q)->print(); //OK, prints: 0x804a018 ^ 0x804a008 return 0; } Ondrej
Ondrej Certik wrote:
The point is that your Python data types are already doing for you automatically what GiNaCs exes are doing in C++: being a reference to a garbage-collected data type. I don't think it is possible to do without that.
I am not sure I understand. Why cannot we use it like in the following example? I compiled it and it works as expected:
[snip leaky code] Question: Why don't you explicitly free the memory you're explicitly allocating? Answer: Because it is a damn pain in the neck! (And, maybe, because you're not used to, since Python does such things for you.) See, with wrapped objects, there're no such problems. Regards -richy. -- Richard B. Kreckel <http://www.ginac.de/~kreckel/>
Dear Ondrej, On Thu, 3 Aug 2006, Ondrej Certik wrote:
I am not sure I understand. Why cannot we use it like in the following example? I compiled it and it works as expected:
No it doesn't. You should consider getting an education in C++.
//function to do that basic *q=pow(e2,e1);
Now, please turn this into while(true) { basic *q = pow(e2, e1); } and watch you program/computer/any airplane within 20 miles crash. After that read about memory leaks. Best wishes, Chris
Now, please turn this into
while(true) { basic *q = pow(e2, e1); }
and watch you program/computer/any airplane within 20 miles crash. After that read about memory leaks.
I was replying to your part of email which says "It will create a new basic object with only the basic-parts of the power" to show, that it is possible to do it my way. So, is it the garbage collection which is the only reason behind ex? You need to use a garbage collector, if you don't want to delete instances by yourself. Like this: #include <iostream> #include "gc_cpp.h" class basic: public gc //class basic { public: basic() {}; }; you need to inherit basic from "gc". gc_cpp is from http://www.hpl.hp.com/personal/Hans_Boehm/gc/ I tested that your while loop while(true) { basic *q = pow(e2, e1); } doesn't crash the computer. (it does however if you don't inherit basic from gc) So back to the original question: is ex only used to do the garbage collection? The syntax above is imho much less complex than the whole thing with ex, on the other hand, I understand that if you implement garbage collection yourself in ex, you have much bigger control of it, and also (maybe) it is faster. Ondrej
On Fri, Aug 04, 2006 at 11:53:23AM -0400, Ondrej Certik wrote:
You need to use a garbage collector, if you don't want to delete instances by yourself.
[snipped]
Like this:
#include <iostream> #include "gc_cpp.h"
class basic: public gc //class basic { public: basic() {}; };
you need to inherit basic from "gc". gc_cpp is from http://www.hpl.hp.com/personal/Hans_Boehm/gc/
1) I don't really understand why one needs such a GC for simple tree-like data types. 2) As a matter of fact, using general-purpose conservative GC prevents one from solving any real-world problems (see Mathematica). Best regards, Alexei. -- All science is either physics or stamp collecting.
Hi! Sheplyakov Alexei wrote:
2) As a matter of fact, using general-purpose conservative GC prevents one from solving any real-world problems (see Mathematica).
Actually, Mathematica objects are reference counted. (Section A.9.2 "Data Structures and Memory Management" of the Mathematica 5.2 Documentation says, that "Every piece of memory used by Mathematica maintains a count of how many times it is referenced. Memory is automatically freed when this count reaches zero.") Regards -richy. -- Richard B. Kreckel <http://www.ginac.de/~kreckel/>
Dear Ondrej, On Fri, 4 Aug 2006, Ondrej Certik wrote:
You need to use a garbage collector, if you don't want to delete instances by yourself.
Yes, doing garbage collection is a reason to have a type ex. It is indeed possible to use another garbage collector. I'm not sure what it would do to performance. This might even depend on the particular computer algebra problem under considerating. Reference counting certainly is more predictable. Algorithms that would seem to be, say, O(N^6), will perform precisely like that in GiNaC. Another reason to have the type ex is automatic evaluation. If objects are to be made by constructors and after that we are going to pass around pointers to them, I'm not sure who is going to take care of automatic evaluation. You can get around this by declaring for every class a function that returns a basic* (as the "pow" function that you showed). However, that defeats the purpose of making the library simpler. Another nice thing about exes is that you can compare them by using the operator ==. That cannot be done with pointers, because it already means something else. Best wishes, Chris
Yes, doing garbage collection is a reason to have a type ex. It is indeed possible to use another garbage collector. I'm not sure what it would do to performance. This might even depend on the particular computer algebra problem under considerating. Reference counting certainly is more predictable. Algorithms that would seem to be, say, O(N^6), will perform precisely like that in GiNaC.
Another reason to have the type ex is automatic evaluation. If objects are to be made by constructors and after that we are going to pass around pointers to them, I'm not sure who is going to take care of automatic evaluation. You can get around this by declaring for every class a function that returns a basic* (as the "pow" function that you showed). However, that defeats the purpose of making the library simpler.
Another nice thing about exes is that you can compare them by using the operator ==. That cannot be done with pointers, because it already means something else.
This answers my question. Thanks. I am not sure about the performance either, it would have to be tried to see, but I it seems that ex -way is going to be faster. As to the eval(), you are completely right, I am addressing the same issue in my library. Either I can evaluate everything automatically right when I construct the expression, or I need to call eval() manually later. As to the "==", you would have to use either of those 2 ways: basic *p=pow(e1,e2), *q=pow(e1,e2); *p==*q or basic &p=*pow(e1,e2), &q=*pow(e1,e2); p==q Thanks for enlightening me the issue, I can see the reasons behind ex now. So what about changing this in ex.h: /** Lightweight wrapper for GiNaC's symbolic objects. Basically all it does is * to hold a pointer to the other objects, manage the reference counting and * provide methods for manipulation of these objects. (Some people call such * a thing a proxy class.) */ to something like: Lightweight wrapper for GiNaC's symbolic objects. Basically all it does is the garbage collection. It would be possible to use another garbage collector, but we are not sure about the speed and reference counting certainly is more predictable. Another reason for ex is the automatic evaluation, which is triggered, whenever an ex is constructed from a basic-derived object. It also provides methods for manipulation of these objects. Ondrej
Dear Ondrej and others, On Fri, 4 Aug 2006, Ondrej Certik wrote:
So what about changing this in ex.h:
[....]
Yes, it seems to be a good idea to mention that the constructor ex::ex(const basic & other) makes sure that automatic evalution takes place. I'll do that. Best wishes, Chris
On Fri, 4 Aug 2006, Ondrej Certik wrote:
you need to inherit basic from "gc". gc_cpp is from http://www.hpl.hp.com/personal/Hans_Boehm/gc/
The roots of GiNaC date back to 1998. I do not know if the gc library had been available at that time (the oldest files in the gc archive date 1999), at least it was not very common and/or not known to me. So we had to invent our own thing.
So back to the original question: is ex only used to do the garbage collection? The syntax above is imho much less complex than the whole thing with ex, on the other hand, I understand that if you implement garbage collection yourself in ex, you have much bigger control of it, and also (maybe) it is faster.
As already written, assignments to references do not work in C++ as someone familiar with Java, C# or other modern garbage collected languages might expect. Using pointers is too dangerous with respect to memory leaks. Smart pointers (std::auto_ptr) solve the memory leak problem, but were relatively new in C++ at this time and do not support reference counting, which means worse performance. Hence we decided to introduced the ex class, which is in principle a smarter smart pointer and was fully under our control. I think comparison using == and automatic evaluation have been positive side effects of this decision. Regards, Alex -- Alexander Frink E-Mail: Alexander.Frink@Uni-Mainz.DE Institut fuer Physik Phone: +49-6131-3923391 Johannes-Gutenberg-Universitaet D-55099 Mainz, Germany
participants (6)
-
Alexander Frink
-
Chris Dams
-
Ondrej Certik
-
Richard B. Kreckel
-
varg@theor.jinr.ru
-
Vladimir Kisil