How to create a python package and publish it in PyPI

私たちの研究室

https://nisk.doshisha.ac.jp/

アドベントカレンダー18日目〜

https://zenn.dev/nislab/articles/2025-advendcaldenr

How to create a python package and publish it in PyPI

Article by エスカーニョ　マルケス、ユイス

In this article I will show you how I got my python library published in PyPI, the python official repository, so that anyone can install it by just doing pip install my-package. Moreover, I will teach you the basics on how to structure a python library or application and pack it to have your own project published as well.

This post assumes that you are somewhat familiar with the python ecosystem. Therefore, you have used either pip or anaconda to install some python libraries.

I will not cover how to create and use a python virtual enviroment, but I highly recommend it if you plan on creating a python library.

TL;DR:

Use the following structure for your code:

├── LICENCE
├── README.md
├── pyproject.toml
└── src
    ├── __init__.py
    └── my_code.py

Then, write the details of our package in the pyproject.toml file and build it with "python -m build .". After you need to create an account in the PyPI website, which will give you the necessary API keys to publish. Finally you can submit your package and it will be available to the entire world within seconds.

My python project: docat

First and foremost, we need some python project to develop. It can be anything, from a simple hello world to a complex application with graphics and networking. In my case, I created docat. This is a simple library that extracts the text from a document, like a PDF, a text document, a presentation... and merges all the text into a single string. This conversion is necessary for a bigger application I'm working on that relies on full text search and indexing of documents. However, this component can be used in other projects, such as forward information to an AI chatbot or applying statistical analysis on text. Therefore, instead of bundling it with my application, I created it as a library that can be imported in any other python program.

To achieve this goal, docat analizes the file, determines what type of file is through its MIME type, and then uses the appropriate python library to extract the contents of the file. There are already libraries to extract the text from a PDF or from a Word document, and docat makes use of these third party libraries, but extracts the work of deciding which one to use and provides the same result for any supported document.

After coding for a while, I have the following structure:

├── __init__.py
├── docat.py
└── parser
    ├── MSDocParser.py
    ├── MSExcelOldParser.py
    ├── MSExcelParser.py
    ├── MSPowerPointParser.py
    ├── OpenDocumentParser.py
    ├── PDFParser.py
    ├── SVGParser.py
    ├── TextParser.py
    ├── __init__.py
    └── generic.py

Where the entry point is the docat.py file.

After the library code has been written, let's jump on how to create a python package that can be imported in any other python program.

Creating a python package

The first step for creating a python package is to structure the code as I will show you now. There are other combinations, but this is the configuration recommended by the Python Packaging Authority.

├── LICENCE
├── README.md
├── pyproject.toml
└── src
    ├── __init__.py
    ├── docat.py
    └── parser
        ├── MSDocParser.py
        ├── MSExcelOldParser.py
        ├── MSExcelParser.py
        ├── MSPowerPointParser.py
        ├── OpenDocumentParser.py
        ├── PDFParser.py
        ├── SVGParser.py
        ├── TextParser.py
        ├── __init__.py
        └── generic.py

To get this structe from the one in the previous section, I followed this steps:

1. Put all the code in the src folder.
2. Add a pyproject.toml file, which I will explain in detail further in this article.
3. Create a README.md explaining how to use the library, which will appear in the package's page once it gets published.
4. Attach a LICENCE file containing the conditions in which we decide to share our project. In my case I used the Apache License Version 2.0, but you can pick any other open source license that you like.

There are multiple things to unpack in this part. Let's take a deeper look at how to write the pyproject.toml configuration file.

Pyproject.toml

The pyproject.toml is the most important file for creating a python package, as it declares how is your package build. It uses the TOML format, which is very intuitive and easy to write.

I will show you the entire pyproject.toml of docat, and then I will analyze each part in more depth.

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[project]
name = "docat"
version = "1.0.0"
authors = [
  { name="LluísE", email="[e-mail]" },
]
description = "Easy and simple document to plain text tool. Supported formats: doc, docx, xls, xlsx, pdf, and many more!"
readme = "README.md"
requires-python = ">=3.6"
classifiers = [
    "Programming Language :: Python :: 3",
    "License :: OSI Approved :: Apache Software License",
    "Operating System :: OS Independent",
]
dependencies = [
    "docx2txt>=0.8",
    "pypdf==4",
    "python-pptx>=0.6",
    "xlrd>=2.0",
    "ppt2txt>=0.1"
]

[project.urls]
Homepage = "https://github.com/lluises/docat"
Issues = "https://github.com/lluises/docat/issues"

[project.scripts]
docat = "docat:main"

[tool.hatch.build.targets.wheel]
packages = ["src"]

[tool.hatch.build.targets.wheel.sources]
"src" = "docat"

Let's understand what each part of the file does.

build-system

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

The build system selects the tooling used to build and package our python application. There are multiple options, and each has its own configuration parameters, but in this article I will cover the basics with hatchling.

project

[project]
name = "docat"
version = "1.0.0"
authors = [
  { name="LluísE", email="[e-mail]" },
]
description = "Easy and simple document to plain text tool. Supported formats: doc, docx, xls, xlsx, pdf, and many more!"
readme = "README.md"
requires-python = ">=3.6"
classifiers = [
    "Programming Language :: Python :: 3",
    "License :: OSI Approved :: Apache Software License",
    "Operating System :: OS Independent",
]
dependencies = [
    "docx2txt>=0.8",
    "pypdf==4",
    "python-pptx>=0.6",
    "xlrd>=2.0",
    "ppt2txt>=0.1"
]

This section covers the metadata of our library. It contains some general details:

• Name: Your package name, it must not exist in the PyPI repository or you will not be able to register it.
• Version: Pay special attention to the version code you choose, because it is required to follow the PyPA version specification. Basically, use "X.Y.Z" where X is the major version, Y the improvements over that version and Z the minor fixes. Always increase the version before updating your package despite how minor your changes might be.
• Authors: Put your name or alias, but be careful with providing to much personal information. Remember that it will be published, so consider if you want to avoid your personal email address or real name.
• Description: A brief explanation of your program.
• Readme: Pointer to the README.md file. The contents of your README will be displayed in the package page at Pypi and it could very likely be the first documentation that anyone using your library will see. Therefore, try to provide some useful details on how to use your package.
• Requires-python: Declare the minimum python version that can run your code.
• Classifiers: This is a list of tags that will help PyPI classify your package. The 3 provided in this article should be enough for most projects, but change the license according to the one you picked. If your code only runs on certain platforms, like Linux or Windows, then you would want to specify it here as well. However, most python code can run anywhere with python installed. Unless you relay on something provided by the operative system, just leave OS Independent.
  dependencies: List all the libraries that your code relies on. If you are using a virtual environment, you can ask pip to list them for you by running pip freeze.

project.urls

[project.urls]
Homepage = "https://github.com/lluises/docat"
Issues = "https://github.com/lluises/docat/issues"

This is a simple list of the project's websites. In my case you can get the code for docat in my github.

project.scripts

[project.scripts]
docat = "docat:main"

This section is optional. In case of just a python library you will not need it. However, for docat I wanted to also allow users to execute the code directly from the command line by typing docat myfile.pdf.

Inside src/docat.py, I have the following code:

def main():
    """
    CLI main
    """
    # [...]
    
if __name__ == '__main__':
    exit(main())

This code will call the function main when the file is run directly, but it will not call the function if the file is imported in another python code. Then it is declared in the pyproject.toml by specifying command_name = "entry_file:function". In my case, I want the command docat to run the code in docat.py (but the .py is excluded) and call the function main(). You can list more commands with different functions if your applications needs it.

tool.hatch.build.targets.wheel

[tool.hatch.build.targets.wheel]
packages = ["src"]

This is specific to the hatchling build system. It tells what folder to get the code from. Despite src being the standard name used in most software projects, you are free to name your directory something else, and that's why you have to declare it.

tool.hatch.build.targets.wheel.sources

[tool.hatch.build.targets.wheel.sources]
"src" = "docat"

This step is optional, but it will help prevent import resolution problems. With out it, when someone installs your package, it will contain the src folder, and then your code inside. However, src is not a very good name for a library. By adding this parameter, users will get docat instead, which avoids having to keep track in which places to put docat, and which ones put src.docat.

For instance, the command docat will not work properly without this configuration, because instead of running docat:main, we should run src:docat:main. I recommend you to use this parameter in order to save on debugging time.

Build the package

After declaring all the details of the library, it is time to build it.

python -m build .

This will create two files at the dist/ folder: one ".tar.gz" and one ".whl" file. This are python packages that can be installed right away. You can try to install the ".whl" file by running pip install:

pip install dist/docat-1.0.0-py3-none-any.whl

Further testing is always a good idea. After installing the library, it can be used as any other python library. Therefore, try opening a python shell with python and import your package:

import docat

Publishing the package

After we bulided our package, we can proceed to distributing it through PyPI. In order to be allowed to publish a package, first and foremost we must create an account.

1. Create an account

Go through the PyPI account registration registration page and fill the required informaiton.

2. Obtain an API key

With the account active, now we will need an API key. This is not the only method to publish, as PyPI offers some integration with GitHub and other platforms, but by obtaining the API key we can manually upload from the console anytime we want.

In the "Account settings" page there is a section with the ability to generate API keys. Proceed to get your own token and make sure to keep it in a safe place as the website will never show you again the secret code.

3. Upload with twine

We are finally ready to upload the package on PyPI. This step requires the use of twine, a utility to upload to the python repository. If you don't have it installed, you can get it by just running .

pip install twine

And then we upload the package to the world:

twine upload dist/docat-1.0.0*

Your console will prompt you for the API token

Uploading distributions to https://upload.pypi.org/legacy/
Enter your API token:

And if the name has not been previously registered, then your library will be released.

Updates

Whenever you want to release a new update, you simply need to adjust the version number and run the upload command again. Make sure to follow the PyPA version specification for any release.

Testing in TestPyPI

Before releasing in the mainline python repository, it is a good idea to test the distribution of your package. To do so, you can register in TestPyPI. It functions exactly like PyPI, but is a test instance so that any mistakes will not be shiped to all your users.

TestPyPI requires a new account, since it is a completely separate instance from the main PyPI repository.

In order to upload your package using twine, you will need to specify the use of testpypi with the -r parameter:

twine upload -r testpypi dist/docat-1.0.0*

Conclusion

To sum up, in this article we learned how to structure our python source code in order to pack it. Moreover, we also learned how to build it and upload it to the PyPI repositories so that anyone can easily find it and install it in their project.

The end

Article by エスカーニョ　マルケス、ユイス
▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁
░▇▇▇▇▇▇▇▇▇░░░░░░░░░░░░░▇▇▇░░░▇▇▇▇▇▇▇▇▇▇▇░
░░░░░░░░▇▇░░░░░░░░▇▇▇▇▇░░░░░░░░░░░░░░▇▇░░
░░░░░░░░▇▇░░░░░▇▇▇░░▇▇░░░░░░░░░░░░░▇▇░░░░
░░░░░░░░▇▇░░░░░░░░░░▇▇░░░░░░░░░░░▇▇▇░░░░░
░░░░░░░░▇▇░░░░░░░░░░▇▇░░░░░░░░░▇▇░░░▇▇░░░
░▇▇▇▇▇▇▇▇▇▇▇░░░░░░░░▇▇░░░░░░░▇▇░░░░░░░▇▇░
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔