Menu

Subscribe

Structure a large Flask API backend App

Intro

Flask is a minimalistic framework that doesn't provide an official way for organizing the application. In this post, I am going to show how we organize our DoubleDibz application directory . Our layout is based on fbone and there are other useful resources on this subject (1 , 2).

What makes DoubleDibz unique is that our backend is purely a API engine that powers the client. This simplifies the complexity in the backend by removing all the logic needed for templating. The backend is only concerned with receiving data, validating data, fetching new data from the db, performing some transformation, and finally returning the result to the client which will decide how to make use of it. If you are interested in creating a fully dynamic web client powered by Flask, read on!

Let's get started

As the home page shows, a hello-world Flask application is extremely simple. 

// run.py
from flask import Flask  
app = Flask(__name__)

@app.route("/")
def hello():  
    return "Hello World!"

if __name__ == "__main__":  
    app.run()

This code is all nice and dandy but is only appropriate for a trivial application complicated enough to fit in one file. For a large Flask application, we need a more sophisticated structure for the growing complexity. We need to manage more complicated routing for various endpoints, being able to configure under different environments (staging vs. prod), abstracting out common logic, etc. Organizing the application in smaller modules goes a long way. It is like writing clean and modular code: both are simpler to understand and easier for other to maintain and extend.

Application layout

our Flask application is based on fbone with some tweaks. The biggest difference here is that the backend only serves as an API layer and has very minimal usage of templating. This is a generic look on our file layout:

├── app
│   ├── __init__.py
│   ├── api      
│   │   ├── __init__.py
│   │   ├── auth.py
│   │   └── ...
│   ├── app.py
│   ├── common
│   │   ├── __init__.py
│   │   ├── constants.py
│   │   ├── helpers.py
│   │   ├── response.py
│   │   └── ...
│   ├── config.py
│   ├── extensions.py
│   ├── frontend
│   │   ├── __init__.py
│   │   └── controllers.py
│   ├── static
│   │   └── ...
│   ├── templates
│   │   ├── app.html
│   │   └── ...
│   └── users
│       ├── UserConstants.py
│       ├── UserForms.py
│       ├── UserHelpers.py
│       ├── UserModels.py
│       └── __init__.py
├── alembic
|   ├── version
│   └── ...
├── tests
│   └── ...
├── fabfile.py
├── manage.py
├── requirements.txt
└── run.py

This may seem like a complicated mess. Lets take a look and see what each folder (or file) is used for.

app/

First and foremost, all the application logic lives under the app/ folder.

  • app/app.py is the main entry point of the Flask application.
  • app/config.py will be storing all the module configuration for different environments (local, staging and prod). You don't want to be messing with production data while developing locally! I'll talk more about how to setup different environment in later posts.
  • app/extensions.py contains declaration of all the flask extensions
  • app/static/ is the folder from which Flask will serve static files. If you want to use a different folder, see here
  • app/common/ consists of common classes and functions that are useful for all the modules. Some examples are global constants, global helper functions.
  • app/api/ contains all the controllers for every API endpoints, such as authentication (login/signup).
  • app/frontend: as opposed to the API folder, the controllers here are serving non API endpoints. For example, it may define entry point to load the HTML file for your fully dynamic javascript application built in backbone, angular, react, or whatever fancy framework you indulge on.
  • app/[module]/ represents a functional area in the application, such as users, posts, chat, and search.
    • app/[module]/[moduleModels] : database models (tables) declaration with Flask-SQLAlchemy
    • app/[module]/[moduleForms]: WTForm definition in Flask-WTF. Useful for form validation.
    • app/[module]/[moduleConstants] and app/[module]/[moduleHelpers] defines the necessary helper functions and constants for this module.

And stuff outside of app/?

The file outside of the application folder shouldn't contain any core business logic for your applications. Those files and scripts are generally for configuring and managing the application. Here I'm listing the scripts and libraries that I use for DoubleDibz.

  • requirements.txt contains all the python dependencies for this application. Install using pip. 
pip install -r requirements.txt`
  • manage.py contains useful scripts (run, initdb, testing, list routes, etc) using Flask-Script
  • run.py will be used to launch the web server locally.
  • fabfile.py: defining shell operations using Fabric (which is amazing). You can define command-line scripts for remote deployment (though it's generally not a good practice if you have multiple servers).
  • tests/ : how can any application be created without any testing? With Flask-Testing, it is simple to write unit tests for models or integration tests for the different API endpoints.
  • alembic/ : folder containing database migration information for SQLAlchemy using Alembic.

We are organizing the code based on different functional areas. Say you want to build a user authentication system in your new Uber for food website, you would group the models, forms, constants, and helper functions logic under the same folder. This structure makes it very clear that all the files under users/ are related for the same purpose. Understanding how authentication and signup works is as easy as browsing the files in this directory. Furthermore, you can expose certain objects and functions from this folder for the consumptions of other modules.

You may have noticed how we add a prefix User-' to the files in the users module, as opposed to simply using models or forms names. This was done deliberately to add more clarity to the meaning of the file just from the file name. Imagine you have a couple different modules, and you open some of the models file in your editor. It is unclear whether a how a models file is different from another models file without examining the code or the file path. We tried our best to set up a naming convention for files, functions and classes in our code in order to provide the most clarity even if it seems cumbersome.

And finally, I mentioned a bunch of Flask extensions and libraries earlier. For clarity sake, this is a list of them:

Though it's outside the scope of this post, I also want to list a couple other extensions and libraries that made my life so much easier.

Final note on styling and consistency

I think one of the most important thing about a code base is consistency. Sometimes, it's more important to be consistent with the rest of the code base than to do what is considered the right convention. Generally, this is what I follow:

  • 2 space indent (no tab)
  • prefer longer names for the sake of clarity and readability (UserModels over models)
  • single quotation 'string' for string
  • Naming convention: module_name, ClassName, method_name, function_name, CONSTANT_NAME, local_variable_name

Of course, I don't always follow my own convention, especially when I'm the only one pushing code. Even so, following some set of rules would guarantee some level of quality and makes it easier collaborating with others on the same code base. But most importantly, consistency is king. it's better to stick with whatever is already there than creating a hot mess.