Fwd: Add method ex::symbols for obtaining all symbols appearing in an expression
On ma, aug 13, 2012 at 9:02 , Richard B. Kreckel <kreckel@ginac.de> wrote:
Wouldn't it be better to go one step further and cache those variables when the variables() function is first used? This avoids quite a lot of overhead in space and time! And then, variables() wouldn't have to be a member function, right?
That is probably best, especially in the light of the (x+y) - y example that Alexei mentioned. I could easily do that in Sage, but I think it wouldn't help in my case because I'm constructing so many expressions. However, I'm thinking it may make sense to do that in GiNaC, because from what I've seen it has some sort of a copy-on-right mechanism for subexpressions? As in: f = sin(x+y) g = f * cos(x-y) h = f * tan(z) will make g and h share the instance (and therefore the cache!) for f? Please correct me if I've misunderstood that. But if it has, this is abstracted away in Sage, so caching the variables() function in Sage would have much less benefit than caching it in GiNaC. I'm not sure whether this will actually give me any speed benefit in practice, because even though I'm using very similar subexpressions over and over again, I'm not sure if I could construct them as the same object without writing really awkward code. I'm interested in your opinions, though. So my next question is: would you accept a patch implementing a recursive, cached version of symbols(), for Sage to call directly, and then have responsibility for optimizing / caching that going to the level of GiNaC? I would normally think that it would be a method of ex (and basic). Where would you say the cache lives if it where a global function? Timo
On 08/13/2012 10:31 AM, Timo Kluck wrote:
On ma, aug 13, 2012 at 9:02 , Richard B. Kreckel<kreckel@ginac.de> wrote:
Wouldn't it be better to go one step further and cache those variables when the variables() function is first used? This avoids quite a lot of overhead in space and time! And then, variables() wouldn't have to be a member function, right?
That is probably best, especially in the light of the (x+y) - y example that Alexei mentioned.
I could easily do that in Sage, but I think it wouldn't help in my case because I'm constructing so many expressions.
However, I'm thinking it may make sense to do that in GiNaC, because from what I've seen it has some sort of a copy-on-right mechanism for subexpressions? As in:
f = sin(x+y) g = f * cos(x-y) h = f * tan(z)
will make g and h share the instance (and therefore the cache!) for f?
Yes, that is right.
Please correct me if I've misunderstood that. But if it has, this is abstracted away in Sage, so caching the variables() function in Sage would have much less benefit than caching it in GiNaC.
I'm not sure whether this will actually give me any speed benefit in practice, because even though I'm using very similar subexpressions over and over again, I'm not sure if I could construct them as the same object without writing really awkward code. I'm interested in your opinions, though.
Hmm, note that even if you cannot construct them as one object they might end up as one object after a while (c.f. the private ex::share(ex) method). Whether that helps or doesn't help in your case depends on the actual use pattern. There remains Alexei's worry that this is likely to substantially slow down all applications. Optimization is full of surprises. In any case, you should experiment with different strategies and carefully observe their impact!
So my next question is: would you accept a patch implementing a recursive, cached version of symbols(), for Sage to call directly, and then have responsibility for optimizing / caching that going to the level of GiNaC?
Well, any patch would have to go into Pynac, in order to be of advantage in Sage. Pynac is a fork of GiNaC. (In the past, we've repeatedly fixed Sage bugs in GiNaC and suggested patches for Pynac and vice-versa, but that doesn't mean that changes in GiNaC propagate automatically into Sage. At least I'm not aware of that.)
I would normally think that it would be a method of ex (and basic). Where would you say the cache lives if it where a global function?
It should be possible to equip that global function with a static hashmap ex -> symbols. Of course, that opens a memory leak because the map is never cleaned up. So, as an alternative, one could maintain that map only while it can really speed up things: store it in an object and delete it when that object is destructed. Cheers -richy. -- Richard B. Kreckel <http://www.ginac.de/~kreckel/>
There remains Alexei's worry that this is likely to substantially slow down all applications. I think his worry related to my initial suggestion for doing this at construction time. I think that just adding a member function implementing its own cache shouldn't add overhead to applications not using it. (The only difference would be the constructor having to initialize the cache to NULL).
Well, any patch would have to go into Pynac, in order to be of advantage in Sage. Pynac is a fork of GiNaC. (In the past, we've repeatedly fixed Sage bugs in GiNaC and suggested patches for Pynac and vice-versa, but that doesn't mean that changes in GiNaC propagate automatically into Sage. At least I'm not aware of that.) Thanks for pointing that out; I was under the impression that Pynac was a Python wrapper for Ginac. I might just have to look at Pynac
2012/8/14 Richard B. Kreckel <kreckel@ginac.de>: then. Do you happen to know what the reason was for forking?
I would normally think that it would be a method of ex (and basic). Where would you say the cache lives if it where a global function? It should be possible to equip that global function with a static hashmap ex -> symbols. Of course, that opens a memory leak because the map is never cleaned up. So, as an alternative, one could maintain that map only while it can really speed up things: store it in an object and delete it when that object is destructed. There would be no memory leak if the cache were to live in the object.
Hi! On 08/14/2012 11:00 AM, Timo Kluck wrote:
2012/8/14 Richard B. Kreckel<kreckel@ginac.de>:
There remains Alexei's worry that this is likely to substantially slow down all applications. I think his worry related to my initial suggestion for doing this at construction time. I think that just adding a member function implementing its own cache shouldn't add overhead to applications not using it. (The only difference would be the constructor having to initialize the cache to NULL).
That is correct. How would you manage that list of symbols? One may have to take care about when to destroy it, in order not to break refcounting and create a memory leak, I suppose.
Do you happen to know what the reason was for forking?
Sage already had its own number system and didn't want to bring another one (CLN). Also, there wasn't much interest in some of the special algebra for high energy physics. No worries.
There would be no memory leak if the cache were to live in the object.
That's correct, too. Still, you should carefully benchmark your proposal to see if it really helps. If it does, and if a .symbols() MF is really a bottleneck in some applications, then, well, why not add it to GiNaC? And, please, discuss this with the Pynac developers, too, for the obvious reasons. Cheers -richy. -- Richard B. Kreckel <http://www.ginac.de/~kreckel/>
Hi, sorry for coming into the discussion so late. I haven't been checking the ginac lists lately. On Wed, 22 Aug 2012 00:51:39 +0200 "Richard B. Kreckel" <kreckel@ginac.de> wrote:
Hi!
On 08/14/2012 11:00 AM, Timo Kluck wrote:
2012/8/14 Richard B. Kreckel<kreckel@ginac.de>:
There remains Alexei's worry that this is likely to substantially slow down all applications. I think his worry related to my initial suggestion for doing this at construction time. I think that just adding a member function implementing its own cache shouldn't add overhead to applications not using it. (The only difference would be the constructor having to initialize the cache to NULL).
That is correct.
How would you manage that list of symbols? One may have to take care about when to destroy it, in order not to break refcounting and create a memory leak, I suppose.
Do you happen to know what the reason was for forking?
Sage already had its own number system and didn't want to bring another one (CLN). Also, there wasn't much interest in some of the special algebra for high energy physics. No worries.
As Richard said, since libraries like MPIR, MPFR, which are already in Sage provide arbitrary precision numbers, we didn't want to introduce a new library for this purpose. Pynac can wrap an arbitrary Python object in the numeric class. This flexibility is not exploited/exposed fully in Sage actually. We are interested in the "high energy physics" stuff. There is just not enough time to write interfaces to GiNaC functionality from Sage. There were a few questions about accessing the Clifford algebra stuff from Sage.
There would be no memory leak if the cache were to live in the object.
That's correct, too.
Still, you should carefully benchmark your proposal to see if it really helps. If it does, and if a .symbols() MF is really a bottleneck in some applications, then, well, why not add it to GiNaC?
And, please, discuss this with the Pynac developers, too, for the obvious reasons.
Timo, if you want to provide a patch to pynac for this purpose, I'd be happy to merge it. I am assuming that if caching is done properly, there won't be any performance regressions. Now we have the tools to test for those, though the test suite needs some love: http://titusnicolae.blogspot.de/2012/06/benchmarking.html Then we can try to push your patch upstream to GiNaC. Cheers, Burcin
participants (3)
-
Burcin Erocal
-
Richard B. Kreckel
-
Timo Kluck