Structure a large Flask API backend App
Flask is a minimalistic framework that doesn't provide an official way for organizing the application. In this post, I am going to show how we organize our DoubleDibz application directory . Our layout is based on fbone and there are other useful resources on this subject (1 , 2).
What makes DoubleDibz unique is that our backend is purely a API engine that powers the client. This simplifies the complexity in the backend by removing all the logic needed for templating. The backend is only concerned with receiving data, validating data, fetching new data from the db, performing some transformation, and finally returning the result to the client which will decide how to make use of it. If you are interested in creating a fully dynamic web client powered by Flask, read on!
Let's get started
As the home page shows, a hello-world Flask application is extremely simple.
// run.py from flask import Flask app = Flask(__name__) @app.route("/") def hello(): return "Hello World!" if __name__ == "__main__": app.run()
This code is all nice and dandy but is only appropriate for a trivial application complicated enough to fit in one file. For a large Flask application, we need a more sophisticated structure for the growing complexity. We need to manage more complicated routing for various endpoints, being able to configure under different environments (staging vs. prod), abstracting out common logic, etc. Organizing the application in smaller modules goes a long way. It is like writing clean and modular code: both are simpler to understand and easier for other to maintain and extend.
our Flask application is based on fbone with some tweaks. The biggest difference here is that the backend only serves as an API layer and has very minimal usage of templating. This is a generic look on our file layout:
├── app │ ├── __init__.py │ ├── api │ │ ├── __init__.py │ │ ├── auth.py │ │ └── ... │ ├── app.py │ ├── common │ │ ├── __init__.py │ │ ├── constants.py │ │ ├── helpers.py │ │ ├── response.py │ │ └── ... │ ├── config.py │ ├── extensions.py │ ├── frontend │ │ ├── __init__.py │ │ └── controllers.py │ ├── static │ │ └── ... │ ├── templates │ │ ├── app.html │ │ └── ... │ └── users │ ├── UserConstants.py │ ├── UserForms.py │ ├── UserHelpers.py │ ├── UserModels.py │ └── __init__.py ├── alembic | ├── version │ └── ... ├── tests │ └── ... ├── fabfile.py ├── manage.py ├── requirements.txt └── run.py
This may seem like a complicated mess. Lets take a look and see what each folder (or file) is used for.
First and foremost, all the application logic lives under the
app/app.pyis the main entry point of the Flask application.
app/config.pywill be storing all the module configuration for different environments (local, staging and prod). You don't want to be messing with production data while developing locally! I'll talk more about how to setup different environment in later posts.
app/extensions.pycontains declaration of all the flask extensions
app/static/is the folder from which Flask will serve static files. If you want to use a different folder, see here
app/common/consists of common classes and functions that are useful for all the modules. Some examples are global constants, global helper functions.
app/api/contains all the controllers for every API endpoints, such as authentication (login/signup).
app/[module]/represents a functional area in the application, such as users, posts, chat, and search.
app/[module]/[moduleModels]: database models (tables) declaration with Flask-SQLAlchemy
app/[module]/[moduleForms]: WTForm definition in Flask-WTF. Useful for form validation.
app/[module]/[moduleHelpers]defines the necessary helper functions and constants for this module.
And stuff outside of
The file outside of the application folder shouldn't contain any core business logic for your applications. Those files and scripts are generally for configuring and managing the application. Here I'm listing the scripts and libraries that I use for DoubleDibz.
requirements.txtcontains all the python dependencies for this application. Install using pip.
pip install -r requirements.txt`
manage.pycontains useful scripts (run, initdb, testing, list routes, etc) using Flask-Script
run.pywill be used to launch the web server locally.
fabfile.py: defining shell operations using Fabric (which is amazing). You can define command-line scripts for remote deployment (though it's generally not a good practice if you have multiple servers).
tests/: how can any application be created without any testing? With Flask-Testing, it is simple to write unit tests for models or integration tests for the different API endpoints.
alembic/: folder containing database migration information for SQLAlchemy using Alembic.
We are organizing the code based on different functional areas. Say you want to build a user authentication system in your new Uber for food website, you would group the
helper functions logic under the same folder. This structure makes it very clear that all the files under
users/ are related for the same purpose. Understanding how authentication and signup works is as easy as browsing the files in this directory. Furthermore, you can expose certain objects and functions from this folder for the consumptions of other modules.
You may have noticed how we add a prefix
User-' to the files in the
users module, as opposed to simply using
forms names. This was done deliberately to add more clarity to the meaning of the file just from the file name. Imagine you have a couple different modules, and you open some of the models file in your editor. It is unclear whether a how a
models file is different from another
models file without examining the code or the file path. We tried our best to set up a naming convention for files, functions and classes in our code in order to provide the most clarity even if it seems cumbersome.
And finally, I mentioned a bunch of Flask extensions and libraries earlier. For clarity sake, this is a list of them:
Though it's outside the scope of this post, I also want to list a couple other extensions and libraries that made my life so much easier.
- redis: in-memory store. Very useful for caching purposes.
- Celery: setting up async jobs
- Pillow: PIL fork. useful for image manipulation like resizing and cropping.
- boto: python interface to Amazon webservice. Useful for uploading files in S3.
- facebook-sdk : Facebook Platform Python SDK
Final note on styling and consistency
I think one of the most important thing about a code base is consistency. Sometimes, it's more important to be consistent with the rest of the code base than to do what is considered the right convention. Generally, this is what I follow:
- 2 space indent (no tab)
- prefer longer names for the sake of clarity and readability (
- single quotation
- Naming convention:
Of course, I don't always follow my own convention, especially when I'm the only one pushing code. Even so, following some set of rules would guarantee some level of quality and makes it easier collaborating with others on the same code base. But most importantly, consistency is king. it's better to stick with whatever is already there than creating a hot mess.