Odd PYPY timings with Offset

[handloomweaver]

Hi,

I have been doing some very simple testing of Offset - namely just creating small functions or generator functions and comparing their performance as offset 'goroutines' compared to normally. But while CPython seems to yield real benefit in reduced timings of up to a third over normal execution I am having wildly odd PYPY results. Take the example below:

import os
from offset import go, maintask, run

def etime():
    """For timing"""
    user, sys, chuser, chsys, real = os.times()
    return user + sys

def f(x):
    next(i for i in xrange(10000) if i >= x)

@maintask
def main():
    start = etime()
    for x in xrange(90000):
        go(f, 1000)
    end = etime()
    elapsed = end - start
    print "offset...." + str(elapsed)

    start = etime()
    for x in xrange(90000):
        f(1000)
    end = etime()
    elapsed = end - start
    print "non offset...." + str(elapsed)

if __name__ == "__main__":
    run()

For CPython I get results like:
offset....3.44
non offset....5.13

So under CPython offset speeds up the execution by almost 2 seconds which is frankly thrilling

However, under PYPY 2.1 the offset timings are terrible, surely indicating that something is wrong (either with my example code) or with offset (i.e I submit this fully expecting that someone will say that this is because my code sucks). But just in case it isn't and it is a bug this is what I get with PYPY:

offset....181.5
non offset....0.77

181 seconds to 0.77!

Thoughts?

[benoitc]

hmmm It looks like the deque implementation on pypy is slower than the one
in python 3.

When running the go function it create the coroutine and place it in the
runq. The only bottleneck here seems the deque implementation in pypy.

I will check if there is a better implementation for pypy to handle the run
queue.

Thanks for your mail anyway.

• benoit

On Tue, Sep 24, 2013 at 4:00 PM, handloomweaver notifications@github.comwrote:

Hi,

I have been doing some very simple testing of Offset - namely just
creating small functions or generator functions and comparing their
performance as offset 'goroutines' compared to normally. But while CPython
seems to yield real benefit in reduced timings of up to a third over normal
execution I am having wildly odd PYPY results. Take the example below:

import os
from offset import go, maintask, run

def etime():
"""For timing"""
user, sys, chuser, chsys, real = os.times()
return user + sys
def f(x):
next(i for i in xrange(10000) if i >= x)
@maintaskdef main():
start = etime()
for x in xrange(90000):
go(f, 1000)
end = etime()
elapsed = end - start
print "offset...." + str(elapsed)

start = etime()
for x in xrange(90000):
    f(1000)
end = etime()
elapsed = end - start
print "non offset...." + str(elapsed)

if name == "main":
run()

For CPython I get results like:
offset....3.44
non offset....5.13

So under CPython offset speeds up the execution by almost 2 seconds which
is frankly thrilling

However, under PYPY 2.1 the offset timings are terrible, surely indicating
that something is wrong (either with my example code) or with offset (i.e I
submit this fully expecting that someone will say that this is because my
code sucks). But just in case it isn't and it is a bug this is what I get
with PYPY:

offset....181.5
non offset....0.77

181 seconds to 0.77!

Thoughts?

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/8
.

[benoitc]

actually this is not deque but the creation of the process. I filled an
issue to pypy:

https://bugs.pypy.org/issue1614

In the mean time I am trying to see what is happening.

On Tue, Sep 24, 2013 at 4:00 PM, handloomweaver notifications@github.comwrote:

Hi,

I have been doing some very simple testing of Offset - namely just
creating small functions or generator functions and comparing their
performance as offset 'goroutines' compared to normally. But while CPython
seems to yield real benefit in reduced timings of up to a third over normal
execution I am having wildly odd PYPY results. Take the example below:

import os
from offset import go, maintask, run

def etime():
"""For timing"""
user, sys, chuser, chsys, real = os.times()
return user + sys
def f(x):
next(i for i in xrange(10000) if i >= x)
@maintaskdef main():
start = etime()
for x in xrange(90000):
go(f, 1000)
end = etime()
elapsed = end - start
print "offset...." + str(elapsed)

start = etime()
for x in xrange(90000):
    f(1000)
end = etime()
elapsed = end - start
print "non offset...." + str(elapsed)

if name == "main":
run()

For CPython I get results like:
offset....3.44
non offset....5.13

So under CPython offset speeds up the execution by almost 2 seconds which
is frankly thrilling

However, under PYPY 2.1 the offset timings are terrible, surely indicating
that something is wrong (either with my example code) or with offset (i.e I
submit this fully expecting that someone will say that this is because my
code sucks). But just in case it isn't and it is a bug this is what I get
with PYPY:

offset....181.5
non offset....0.77

181 seconds to 0.77!

Thoughts?

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/8
.

[benoitc]

fixed in cbe97d8 . Thanks!

[benoitc]

btw a better test may be synchronizing the goroutines:

import os
from offset import go, maintask, run
from offset.sync import WaitGroup

def etime():
    """For timing"""
    user, sys, chuser, chsys, real = os.times()
    return user + sys

def f(x):
    next(i for i in range(10000) if i >= x)

def fc(x, wg1):
    wg1.wait()
    f(x)

@maintask
def main():
    wg1 = WaitGroup()
    wg1.add(90000)

    start = etime()
    for x in range(90000):
        go(fc, 1000, wg1)

    end = etime()
    elapsed = end - start
    print("offset queued...." + str(elapsed))

    start = etime()
    for i in range(90000):
        wg1.done()

    end = etime()
    elapsed = end - start
    print("offset done...." + str(elapsed))


if __name__ == "__main__":
    start = etime()
    run()
    end = etime()
    elapsed = end - start
    print("offset...." + str(elapsed))

    start = etime()
    for x in range(90000):
        f(1000)
    end = etime()
    elapsed = end - start
    print("non offset...." + str(elapsed)))

[benoitc]

with above test:

Pypy:

$ python ../test.py 
offset queued....0.45
offset done....0.02
offset....1.02
non offset....0.28

Python 3:

$ python ../test.py 
offset queued....1.78
offset done....0.10999999999999988
offset....7.55
non offset....4.98

This is quite normal that the offset version is slower in such test since there is no concurrence handled there, so you pay the price of switching to another routine.