Extract captions from this video
Closed this issue · 2 comments
github-actions commented
Captions for PyGotham TV 2021 are sponsored by Two Sigma
- Hello, PyGotham.
My name is Simon, and
in just 10 minutes, how to build,
test and publish an open
I'm actually going to publish
a library while I record this talk.
This is a piece of code,
which I've been copying
for over a decade now.
It's a very simple little thing
that helps you create string based
shortened versions of numeric identifiers.
I'll do a quick demonstration
of what that looks like.
Let's paste it into a Jupyter notebook.
And now if I do pid.from_int (1234),
it says, gxd.
And if I do pid.to_int
So very simple piece of code.
Let's turn this into a package
on the Python package on
So I'm going to create
I checked, and this is an
And I'm going to add that file.
Let's create a new file called pid.py.
We'll stick it in that folder
So that's my code.
Creating a package, all you need for sure
is a setup.py file,
which describes how the package will work.
So I'm going to create that.
And then I'm going to paste
in this little template here.
This is the simplest possible version
of a valid set up .py file.
I'm going to give it the
it's a tiny Python library
And the only module
that pids.py file I just created.
So now in here we have two files, pid.py,
and setup.py.
Now, if I run python3 setup.py sdist,
this is where the magic happens,
that command back created me
a file called pids-0.1.tar.gz.
And that is a file,
which I can install into any environment
to install this package.
Can actually demonstrate that once again,
in this Jupyter notebook,
let's pop that open and
paste that, what was it
When that processes it, it installs it.
And now if I type import pids,
I have pids.pid as a piece of code
that I can start to run.
So let's upload this to the
where we do that is with
I can say twine upload
the name of that file
When I do this, it'll ask me to log in
with my username and my PyPI password.
So you'll need to create an account
on PyPI before you do this.
I do that.
It shows me the upload
And this is live on
This is now a package which anyone
in the world can install by
But you'll notice it says here,
the author of this package
has not provided a project description.
We're missing our documentation.
So let's build that
that we can add into that package.
So here's a tiny bit of
I'm going to save that in
and VS Code can give us a little preview
of it, showing you what that looks like.
Not very exciting,
but it does explain what's going on.
So now we need to bundle
that into our overall package and
the why we do that is by adding
a tiny bit more of a recipe
Here we go. I'll copy and paste this in.
So we can add two more arguments here.
There's description, long description
and long description, content type.
And here we're doing,
we're calling a function I've written
called get long description,
which reads that weekly file on disc
and puts that in a variable.
So we add this a little
and we set that as the long description,
I'm going to bump up the
to reflect that change.
And now once again,
we can run python3 setup.py sdist.
Oh, it doesn't like that 'cause
I forgot the input OS there.
Could do that.
We now have a file in
and again, if we do twine
And stick in those
This has given us our
and this release, it's improved.
It's got documentation.
The contents of that readme
is now available for people to see.
So for our next step,
I'm going to need a tiny bit
I need to add pytest as a test dependency.
So I'll paste this in here.
This is saying that the
are pytest, and now I'm
a Python virtual environment.
I'll call it venv.
I'll do source venv/bin/activate
And I'm going to type pip installe -e,
editable of my current directory
and it's test dependencies.
So running this installs pytest.
Now, if I run pytest, it runs my test,
except I don't have any tests yet.
So let's create a test folder,
and I'm going to create a file
in that called test_pids.py.
And the great thing about pytest is
that tests are really,
So let's say a test_from_int():
and it's going to say
assert pids.pid.from_int(1234)=="mzq"
So I'll try running that test.
And it fails because I
is the correct value.
So I can stick gxd in
I'll do that in the opposite direction.
So I'll test test_to_int("gsd") == 1234
And when that tests there,
And so now I have a passing test suite,
and this can obviously
be extended to be a lot more thorough.
So the next step is to
I'm going to create a new
I'll use the default settings for that.
And then handy thing about GitHub
is it just gives you the commands
you need to run for you
So I'll do that.
I'm going to git add my README,
my pids.py, my setup.py,
And then I'll paste in
and that will go ahead
and create that repository for me.
But the really neat thing about GitHub
these days is you can use GitHub
actions to run your tests for you.
The way you do that is be by creating
a Github workflows directory.
So rather than create that by hand,
I'm going to copy that
from a project I worked on recently,
My sqlite explain project
I happen to know has one of these.
And I'm gonna copy that folder here.
So now, if we look in here,
we now have a GitHub folder
with a publish.yml and a test.yml.
This right here is the one
What this does is it installs the package
under four different versions of Python,
loads that and runs those tests for us.
So I can now do git add.github
git commit, GitHub Actions, git push.
And over here, GitHub will spot that
those actions have been
that test action for me straight away.
You can see that this actually
turns into four different
One for each of those versions of Python
that we are targeting,
and it's already started running those.
It's running them in parallel,
all different four version.
We look at this one here,
it's set up some caching
It's run the tests and they've passed
and we can see the
And just like that, our test has been run
on four versions of Python and our
comment there even gets
to show what had happened.
So I have one last trick up my sleeve.
I'm going to push a new release directly
from GitHub and without running anything
on my computer at all.
Where I'm going to do this is by creating
a API token on PyPI called pids.
I'm gonna give it this scope
of being able to publish
and that token,
and I will copy this token and then paste
it into the Secrets for
So I can do, add secret,
call it PYPI token and paste it in here.
And now anytime I push a release
to this GitHub repository,
my published work flow
will run more published that release.
So I'm gonna modify my version number.
Let's call this 0.1.2,
And I'm going to publish a new release.
It's create new release.
It's going to be called 0.1.2.
And let's just say,
I hit the publish button.
And once again, that
So this is my published
which is running the tests first,
because you wouldn't want
to publish your package if you
haven't thoroughly tested it.
And once the tests are passed
hit deploy action,
which ones that coin upload scripts
that I showed you earlier.
So this right here is installing
And now it's publishing
that's done. And here is a package
on PyPI that was published entirely
through automation using GitHub actions.
So thanks for listening,
and I've published extensive notes on at
github.com/simonw/pygotham-packaging.
github-actions commented
[youtube] VMnLXynUqys: Downloading webpage
[youtube] VMnLXynUqys: Downloading android player API JSON
[youtube] VMnLXynUqys: Downloading MPD manifest
[youtube] VMnLXynUqys: Downloading MPD manifest
[info] VMnLXynUqys: Downloading subtitles: en-US
[info] VMnLXynUqys: Downloading 1 format(s): 22
[info] Writing video subtitles to: How to build, test and publish an open source Python library [VMnLXynUqys].en-US.ttml
[download] Destination: How to build, test and publish an open source Python library [VMnLXynUqys].en-US.ttml
[download] 1.00KiB at Unknown B/s (00:00)
[download] 3.00KiB at Unknown B/s (00:00)
[download] 7.00KiB at Unknown B/s (00:00)
[download] 15.00KiB at Unknown B/s (00:00)
[download] 23.09KiB at 3.65MiB/s (00:00)
[download] 100% of 23.09KiB in 00:00 at 303.16KiB/s