Django formsets tutorial series: Foundations

Version info

For this tutorial, I used Python version 3.8 and Django version 3.1.

Prerequisites

Before going through this tutorial, you will probably want to have a fair grasp of how Django works, since formsets aren't very beginner-friendly. If you're a complete beginner and don't know much Python, I recommend the Django Girls tutorial and/or the Django for Beginners book. If you know a lot of Python but are new to Django, then you can try the official tutorial. That said, this tutorial is fairly in-depth and comprehensive so buckle up! If you feel that you are quite the savvy Django developer with a keen understanding of vanilla forms, only formsets are a blind spot, you will probably want to scroll through much of the tutorial, looking for parts that seem new to you.

Aim

This tutorial won't teach you everything about formsets. It will however give you a solid foundation to build on (and that future tutorials will build on), where you can be confident you understand what's going on with formsets and you don't have to worry about any "magic" happening behind your back.

Introduction

Django forms (Django docs) are used to allow users to input data and submit them with a request, usually a GET request for search functionality, and a POST request for anything that involves sensitive data. If you don't know much about what the HTTP verbs GET and POST mean, I recommend listening to this podcast episode with Julia Evans. If you want to brush up on forms in Django, you can have a listen to this episode from the Django Riffs podcast, though there will be a fair bit of review in this tutorial. Here, we will use formsets (Django docs) for generating multiple forms of the same type, which is super helpful for allowing users to send in multiple forms at once.

We will start by setting up a minimal django project and playing around with forms in the shell (python manage.py shell), focusing on aspects that are helpful for understanding formsets. We then move on to playing with formsets in the shell. Lastly, we shift to writing code for the project with a simple model, form, formset and template. Hopefully it will all make sense :)

Code

If you want to have a look straight away at the code and files of the simple project we'll be building in this tutorial, you can go to my Django formset tutorials github repo. That said, apart from maybe helping to clarify things like directory structure, the project in and of itself doesn't explain much.

Project setup

As a simple example, let's say our project is built for a team of explorers. The forms will allow the team to send in data about treasures that they find.

First, create a directory that you will use to work in, e. g. a folder on your desktop, and go to it using your platform's command-line interface (explanation of CLI by Django Girls). Here's how I do it on a Mac:

Lowe:Desktop lowe$ mkdir explorers_project
Lowe:Desktop lowe$ cd explorers_project/

Now we want to set up our virtual environment, where we install django. There are different tools that can be used for this. Personally I'm used to pipenv (Real Python tutorial), but venv (Real Python tutorial) or any other virtual environment package should work if you look up the commands. You might notice slight differences in what files are in the project then, like you might have a venv directory which I don't - it's nothing to worry about.

Lowe:explorers_project lowe$ pipenv install django==3.1
Lowe:explorers_project lowe$ pipenv shell

You'll notice that the CLI prompt looks a bit different, e. g. (explorers_project) bash-3.2$.

Now let's create the project (I'm using a period at the end of the command here - otherwise the project is put inside of a explorers_project directory)

(explorers_project) bash-3.2$ django-admin startproject explorer_project .

This gives us the following directory structure

.
├── Pipfile
├── Pipfile.lock
├── explorer_project
│   ├── __init__.py
│   ├── asgi.py
│   ├── settings.py
│   ├── urls.py
│   └── wsgi.py
└── manage.py

Regular forms in the shell

Now, let's look at a simple example of a django form, using the shell.

First we jump into the shell:

(explorers_project) bash-3.2$ python manage.py shell

We'll start by defining a Form subclass. Remember, we want forms for our explorers to submit data about treasures they find. To keep things simple, we assume that the explorers only want to store the name of the treasure and its estimated price.

Note that the official Django docs use >>> symbols to indicate that things are done in the interactive shell, but I'm skipping that here so that it's easier for you to copy-paste the code blocks.

# shell
from django import forms

class TreasureForm(forms.Form):
    name = forms.CharField(max_length=100)
    estimated_price = forms.IntegerField()

We have now created a Form class. We don't have an actual, specific form that we can use in templates or pass data to yet. To get that, we generate instances of our TreasureForm.

# shell
f1 = TreasureForm() # first form
f2 = TreasureForm() # second form

Once created, we can do things like bind data to each form or use them to generate HTML, as we'll see shortly.

Form methods and attributes

Let's consider some of the methods and attributes of our form instances.

.as_p()

You are very likely to have used form.as_p(), or form.as_table() in templates before ({% form.as_p %}). You may or may not have tried out the same thing in the interactive shell. It can however be illuminating.

# shell
print(f1.as_p())

We then get this output

<p><label for="id_name">Name:</label> <input type="text" name="name" maxlength="100" required id="id_name"></p>
<p><label for="id_estimated_price">Estimated price:</label> <input type="number" name="estimated_price" required id="id_estimated_price"></p>

The generated HTML input tags' name attributes correspond directly to what we called the attributes of our TreasureForm. When a user interacts with a form on a Django site and the form data are rolled into a data dictionary for a view to handle, the name attributes also determine what the keys in the data dictionary will be. So if the above form was set to send data as POST requests, and hooked up to a view, then that view could access submitted data with request.POST.get('name') and request.POST.get('estimated_price').

.initial

This attribute shows the initial data of a form, i. e. the data that it holds before any user interaction has happened. If we try this with one of the forms we created earlier

# shell
f1.initial

then we get an empty dictionary ({}). We can however generate a new form using our TreasureForm class, this time with some initial data

# shell
f3 = TreasureForm(initial={'name':'Rock'})

What about now?

# shell
f3.initial
# output: {'name': 'Rock'}

The initial name value is in there as expected. Note how we passed in the initial data when we created the form instance from our TreasureForm class.

What happens if we generate HTML inputs using f3?

# shell
print(f3.as_p())

This time we get

<p><label for="id_name">Name:</label> <input type="text" name="name" value="Rock" maxlength="100" required id="id_name"></p>
<p><label for="id_estimated_price">Estimated price:</label> <input type="number" name="estimated_price" required id="id_estimated_price"></p>

meaning that the name input has an additional attribute value, set to 'Rock'

.data, .is_bound and .has_changed()

We just saw how to give a form some initial data when it's being created. But how do we create a form with 'real' data, that is, data that should be interpreted as user-submitted? The answer is by passing a data argument when creating the form.

f4 = TreasureForm(data={'name':'The Holy Grail', 'estimated_price':'300'}, initial={'name':'Rock'})

Django essentially interprets the above call as meaning that the following has happened:

• the user was shown a HTML form which had
  • an input named 'name' with the initial value of 'Rock'
  • an input named 'estimated_price' with no initial value
• the user then
  • changed the 'name' input's value to 'The Holy Grail'
  • entered '300' for the 'price' input, and
  • submitted the form.

We can now check the data bound to the form:

# shell
f4.data
# outputs {'name': 'The Holy Grail', 'estimated_price': '300'}

Forms are referred to as unbound when they haven't been passed any data that are to be interpreted as user-submitted. So our earlier forms f1, f2 and f3 are all considered unbound. Note that even though f3 was passed a dictionary of initial data, that doesn't count - the initial data aren't interpreted as being user submitted. What you usually want to pass in as initial data is information that is already known. For instance, if our form is used to update an already recorded treasure, we'll want to fill the name field of the form with the treasure's already saved name, and the same thing goes for the estimated_price. That way, the explorer using our form doesn't have to fill in the same information again, like if they want the name to stay the same and only change the estimated price. In cases like these, we want to pass data to the initial parameter of our TreasureForm's constructor when generating forms.

Bound forms are forms like our form f4, which holds data that are to be interpreted as user-submitted. When these forms are generated, data must be passed to the data parameter of the Form class, just like we did with our TreasureForm. There may or may not be data passed to the initial parameter also.

You can easily check if a form is bound or not

# shell
f3.is_bound # False
f4.is_bound # True

When we use forms with views, we often apply this pattern:

• The user makes a GET request
• Inside of the view, an unbound form is generated (with or without initial data)
• The unbound form is put inside of a context dictionary
• The context is used for rendering a template, which includes a HTML form
• The user inputs data and submits them with a POST request
• The data (in the shape of a type of dictionary we call request.POST) are used to create a bound form, while also passing any initial data that were used for the unbound form

Since bound forms hold information about the user's input, we might ask if the user actually changed/input anything prior to submitting. In our case, that means asking if the dictionary(-like) passed to the data parameter of the TreasureForm constructor holds the same set of key-value pairs as the dictionary passed to the initial parameter (initial defaults to an empty dictionary). The .has_changed() method checks exactly this

# shell
f4.has_changed() # True
f5 = TreasureForm(data={}) # implicitly sets `initial` data to {}
f5.has_changed() # False

.is_valid(), .cleaned_data and .errors

If we find that the user has provided input (.has_changed() returns True) , we want to check if the data they have submitted are valid or not. For this, we use .is_valid().

# shell
f4.is_valid() # True
f6 = TreasureForm({'name':'An ex-parrot', 'estimated_price':'Very high'}, initial={'name':'Rock'})
f6.has_changed() #True
f6.is_valid() # False

Note: instead of explicitly passing our data as the data argument, here we used it as a positional argument. Since the first position of the TreasureForm constructor is devoted to the data argument, the end effect is the same.

Why was the f6 form false? Because django validates data based on the type of a field. For an IntegerField instance, django expects the input value to consist of a string that can, predictably, be converted to an integer. So '300' works fine, but 'Very high' does not.

Of note is that .is_valid() doesn't actually just check if the data are valid. The method also triggers the actual conversion/cleaning of data, from raw input strings into appropriate data types based on specified field types. The cleaned data can now be accessed as a dictionary by using .cleaned_data.

# shell
f4.cleaned_data # {'name': 'The Holy Grail', 'estimated_price': 300}
f6.cleaned_data # {'name': 'An ex-parrot'}

As you can see, even if .is_valid() returns False (in the case of f6), the .cleaned_data attribute is still added with whatever data that were valid, while skipping the invalid parts. For fields with invalid data, information is added to the .errors attribute dictionary.

# shell
f4.errors # {} 
f6.errors # {'estimated_price': ['Enter a whole number.']}

If the form has been validated then you can proceed with using the .cleaned_data - and if the form was not valid, in a view you'd usually render it back to the user, where django makes sure to include the generated errors.

Formsets in the shell

Formsets - what are they even good for

Now that we've looked at different parts of Form instances, let's shift our attention to formsets. What can you do if you want to generate a bunch of forms that you can present to the user? You might be tempted to try something like this:

# shell
# this won't do what we want
my_forms = []
for i in range(3):
    form = TreasureForm()
    my_forms.append(form)

What's the problem with this? Maybe you can already guess the answer based on the previous discussion about forms. What do you think the HTML inputs generated by our forms might look like?

# shell
print(my_forms[0].as_p())
print(my_forms[1].as_p())

# output
<p><label for="id_name">Name:</label> <input type="text" name="name" maxlength="100" required id="id_name"></p>
<p><label for="id_estimated_price">Estimated price:</label> <input type="number" name="estimated_price" required id="id_estimated_price"></p>

<p><label for="id_name">Name:</label> <input type="text" name="name" maxlength="100" required id="id_name"></p>
<p><label for="id_estimated_price">Estimated price:</label> <input type="number" name="estimated_price" required id="id_estimated_price"></p>

The generated HTML inputs are exactly the same. If you render this to a template, you will have duplicate input tags. On the surface that might seem to be what you want. But if the user fills out both forms and POSTs their data, nothing fancy happens. The value attached to 'estimated_price' in the request isn't turned into a list holding the values from both inputs with that name. Instead, the last input's value simply overrides whatever came before.

An example might help: Say you render the three forms in my_forms (using a template with a {% for form in my_forms %}... loop). The user will be shown three forms. They input an estimated price value of 100 for the first form's "Estimated price" input, 400 for the second form, and 60 for the third form. When they submit their data, all that your server will receive (and that will be available in the request.POST dictionary) is 'estimated_price': 60. The last input's value overrides the preceding ones.

Formsets to the rescue

So our brute force approach above didn't work. Instead, Django's formsets provide an excellent tool for generating copies of a form on the fly. It's easy to get confused when learning them though, largely because of the way things are named.

*Note: To be fair, if having different names for a set number of forms is the only thing you are worried about, there is an alternative solution, as raised by user DirectDuck in a GitHub issue. When instantiating forms you can specify a 'prefix' parameter, which would solve the specific problem mentioned above. As we'll see however, formsets provide additional benefits.

What's in a name? For usability, a lot

We already talked about how we use a form class to generate form instances, which is what we actually use for generating the html in forms or saving data. For someone not very used to classes or object-oriented programming in general, that might take a bit of getting used to, but it's fairly straightforward. Things get a bit more tricky when we talk about formsets though. Let's look at the general steps for creating and using a formset, one by one:

1. First you need a Form subclass, so TreasureForm in our case
2. You pass the Form (TreasureForm) to the formset_factory function, to create a formset class
3. You instantiate the formset class to get an actual formset
4. The formset has a .forms attribute, which you can loop over to access individual forms.

With a function name like formset_factory, you might expect it to return a formset instance that you can start using right away. But again, note that what it returns is actually a formset class. If you want, you can think of the formset_factory as creating a formset factory (rather than being one itself), i. e. the formset class which is used to "produce" formset instances.

Maybe you don't find it too hard to keep track of what's happening so far. In future tutorials, we'll get to a couple of points that help to explain why things easily get confusing when you're new to formsets.

Creating our first formsets

Let's look at how to actually execute the above steps in the shell, using our TreasureForm class.

# shell
from django.forms import formset_factory
# step 1 we already did earlier, when we defined our TreasureForm class
TreasureFormSet = formset_factory(TreasureForm) # step 2
treasure_formset = TreasureFormSet() # step 3
for form in treasure_formset.forms: # step 4
    print(form.as_p())

That nets us the modest output of

<p><label for="id_form-0-name">Name:</label> <input type="text" name="form-0-name" maxlength="100" id="id_form-0-name"></p>
<p><label for="id_form-0-estimated_price">Estimated price:</label> <input type="number" name="form-0-estimated_price" id="id_form-0-estimated_price"></p>

We only get one measly form. You're probably not impressed. We'll get there.

So what's different about the form generated by our formset? Well, we might for instance look at the name attribute of the generated input for our estimated_price field.

• vanilla form: estimated_price
• formset form: form-0-estimated_price

Though uglier, you might see how the formset form seems to be following a naming convention that avoids the HTML input naming collisions. This is made clearer if we, when building our formset class, set extra=2, meaning that we want two empty (i. e. with no initial data) forms to be included in all formset instances generated using our formset class.

# shell
TreasureFormSet = formset_factory(TreasureForm, extra=2)
treasure_formset = TreasureFormSet()
for form in treasure_formset.forms:
    print(form.as_p())

# output
<p><label for="id_form-0-name">Name:</label> <input type="text" name="form-0-name" maxlength="100" id="id_form-0-name"></p>
<p><label for="id_form-0-estimated_price">Estimated price:</label> <input type="number" name="form-0-estimated_price" id="id_form-0-estimated_price"></p>
<p><label for="id_form-1-name">Name:</label> <input type="text" name="form-1-name" maxlength="100" id="id_form-1-name"></p>
<p><label for="id_form-1-estimated_price">Estimated price:</label> <input type="number" name="form-1-estimated_price" id="id_form-1-estimated_price"></p>

As you might have expected, the generated names for inputs related to estimated_price are form-0-estimated_price and form-1-estimated_price.

So what if you now were to pass the treasure_formset to a context and use it to render a template which has a for loop like {% for form in treasure_formset %}...? To the user, things would look the same as when we used our brute force approach, and so they might input estimated prices of 100 and 400. But this time, when they hit submit, the POST data will have key-value pairs like {... 'form-0-estimated-price': '100', 'form-1-estimated-price': '400' ...}, and in your view, you'll get a request.POST dictionary with those same values. You can then pass the request.POST to your formset class to create a bound formset.

The formsets that bind us

Bound formset? That's right - just like individual forms, formsets can be created with user input data and/or initial data. If they are created with user input, we call them bound.

# shell
treasure_formset_data = {
    'form-0-name': 'An Irish shoebox',
    'form-0-estimated_price': '0',
    'form-1-name': 'The funniest joke in the world',
    'form-1-estimated_price': '10000',
}
TreasureFormSet = formset_factory(TreasureForm)
treasure_formset = TreasureFormSet(treasure_formset_data)
treasure_formset.is_bound # True

What if we try .is_valid()?

# shell
treasure_formset.is_valid()

We get an error:

# output
django.core.exceptions.ValidationError: ['ManagementForm data is missing or has been tampered with']

ManagementForm: The dictator

What's this about ManagementForm? Well, all formsets actually include an additional form, that's hidden from the user and holds metadata about the other forms in the formset. This management form must be included whenever you render a formset to a template. This is because if the data pertaining to the management form isn't included in the HTTP request upon submission (and hence, isn't included in the dictionary you use to create a bound formset), you will get the above ValidationError. Here's an example of HTML generated by the management form:

# shell
treasure_formset = TreasureFormSet()

print(treasure_formset.management_form)

# output
<input type="hidden" name="form-TOTAL_FORMS" value="1" id="id_form-TOTAL_FORMS"><input type="hidden" name="form-INITIAL_FORMS" value="0" id="id_form-INITIAL_FORMS"><input type="hidden" name="form-MIN_NUM_FORMS" value="0" id="id_form-MIN_NUM_FORMS"><input type="hidden" name="form-MAX_NUM_FORMS" value="1000" id="id_form-MAX_NUM_FORMS">

This is what the inputs mean:

• form-TOTAL_FORMS: The total number of forms in the formset
• form-INITIAL_FORMS: The number of forms in the formset that have some initial (non-user-input) data
• form-MIN_NUM_FORMS: The minimum number of forms that the formset must hold (defaults to 0)
• form-MAX_NUM_FORMS: The maximum number of forms that the formset can hold (defaults to 1000)

(all the above counts exclude the management form itself)

Let's see how we can change the formset_factory call in order to modify what management form data will be generated when instantiating formsets.

TreasureFormSet = formset_factory(TreasureForm, extra=3, min_num=2, max_num=42)
treasure_formset = TreasureFormSet()
print(treasure_formset.management_form)

# output
<input type="hidden" name="form-TOTAL_FORMS" value="5" id="id_form-TOTAL_FORMS"><input type="hidden" name="form-INITIAL_FORMS" value="0" id="id_form-INITIAL_FORMS"><input type="hidden" name="form-MIN_NUM_FORMS" value="2" id="id_form-MIN_NUM_FORMS"><input type="hidden" name="form-MAX_NUM_FORMS" value="42" id="id_form-MAX_NUM_FORMS">

We still haven't included any initial data, so form-INITIAL_FORMS stays at a value of 0. We specified the minimum number of forms to be 2 (min_num / form-MIN_NUM_FORMS) and the maximum to be 42 (max_num / form-MAX_NUM_FORMS). We also said that we wanted an extra 3 forms, in addition to the minimum number (or the number of forms with initial data - we'll get to that). So in total, we got 5 forms (form-TOTAL_FORMS). We can also confirm this by checking the number of forms in the formset.

# shell
len(treasure_formset.forms) # outputs 5

Bound formsets: Take two

Now that we've wisened up, let's try creating a bound formset again.

# shell
treasure_formset_data = {
    'form-TOTAL_FORMS': '1',
    'form-INITIAL_FORMS': '0',
    'form-MIN_NUM_FORMS': '0',
    'form-MAX_NUM_FORMS': '1000',
    'form-0-name': 'An Irish shoebox',
    'form-0-estimated_price': '0',
}
TreasureFormSet = formset_factory(TreasureForm)
treasure_formset = TreasureFormSet(treasure_formset_data)
treasure_formset.is_bound # True
treasure_formset.is_valid() # True

It works! This time we included the default values that Django uses for management forms' initial data. Just like with individual forms, once a formset has been validated we can access its cleaned data.

# shell
treasure_formset.cleaned_data
# output: [{'name': 'An Irish shoebox', 'estimated_price': 0}]

Instead of a single dictionary we now get a list of dictionaries, where each dictionary corresponds to one of the formset's forms. Let's try things again, with some more data.

# shell
treasure_formset_data = {
    'form-TOTAL_FORMS': '2',
    'form-INITIAL_FORMS': '0',
    'form-MIN_NUM_FORMS': '0',
    'form-MAX_NUM_FORMS': '1000',
    'form-0-name': 'An Irish shoebox',
    'form-0-estimated_price': '0',
    'form-1-name': 'The funniest joke in the world',
    'form-1-estimated_price': '10000',
}
TreasureFormSet = formset_factory(TreasureForm)
treasure_formset = TreasureFormSet(treasure_formset_data)
treasure_formset.is_valid()
treasure_formset.cleaned_data
# output: [{'name': 'An Irish shoebox', 'estimated_price': 0}, {'name': 'The funniest joke in the world', 'estimated_price': 10000}]

Note that we had to bump the value of form-TOTAL_FORMS, since there were now data from two forms being fed into the formset at instantiation.

If you prefer, you can of course also loop over the forms in the formset and use their .cleaned_data attribute instead

# shell
for form in treasure_formset:
    print(form.cleaned_data)
# {'name': 'An Irish shoebox', 'estimated_price': 0}
# {'name': 'The funniest joke in the world', 'estimated_price': 10000}

Initial Data

Let's generate a formset with some initial data. For forms, the initial data came in the shape of a single dictionary, basically mirroring the request.POST dictionary of user input data. With the FormSet constructor, there is an asymmetry. The request.POST dictionary doesn't change of course, as we saw above, and so the data argument still just takes a dictionary. But for the initial data, the FormSet constructor expects a list of dictionaries.

# shell
treasure_formset_initial_data = [
    {
    'name': 'Toyota AE86',
    'estimated_price': '3000',
    },
    {
    'name': 'Tofu',
    'estimated_price': '5',
    }
]
TreasureFormSet = formset_factory(TreasureForm, extra=1)
treasure_formset = TreasureFormSet(initial=treasure_formset_initial_data)
treasure_formset.initial
# output: [{'name': 'Toyota AE86', 'estimated_price': '3000'}, {'name': 'Tofu', 'estimated_price': '5'}]

What about the HTML it will generate? Instead of printing treasure_formset.management_form.as_p() and then looping over and printing the individual forms, we can actually use a handy shortcut.

# shell
print(treasure_formset.as_p())

# output (with added line breaks for clarity)
<input type="hidden" name="form-TOTAL_FORMS" value="3" id="id_form-TOTAL_FORMS"><input type="hidden" name="form-INITIAL_FORMS" value="2" id="id_form-INITIAL_FORMS"><input type="hidden" name="form-MIN_NUM_FORMS" value="0" id="id_form-MIN_NUM_FORMS"><input type="hidden" name="form-MAX_NUM_FORMS" value="1000" id="id_form-MAX_NUM_FORMS">

<p><label for="id_form-0-name">Name:</label> <input type="text" name="form-0-name" value="Toyota AE86" maxlength="100" id="id_form-0-name"></p>
<p><label for="id_form-0-estimated_price">Estimated price:</label> <input type="number" name="form-0-estimated_price" value="3000" id="id_form-0-estimated_price"></p> 

<p><label for="id_form-1-name">Name:</label> <input type="text" name="form-1-name" value="Tofu" maxlength="100" id="id_form-1-name"></p>
<p><label for="id_form-1-estimated_price">Estimated price:</label> <input type="number" name="form-1-estimated_price" value="5" id="id_form-1-estimated_price"></p> 

<p><label for="id_form-2-name">Name:</label> <input type="text" name="form-2-name" maxlength="100" id="id_form-2-name"></p>
<p><label for="id_form-2-estimated_price">Estimated price:</label> <input type="number" name="form-2-estimated_price" id="id_form-2-estimated_price"></p>

Hopefully you'll be able to roughly understand what's going on, based on what we've discussed so far. If you feel lost, you might want to reread the previous examples of generated HTML.

The formset generates four forms in total:

• A manager form that's hidden (type="hidden") from the user, and which holds metadata about the other forms
• A form with an initial value of "Toyota AE86" for the name field, and an initial value of "3000" for the estimated price field
• A form with initial name value of Tofu, and initial estimated price value of "5"
• An 'extra' form that has no initial values set, since the TreasureFormSet used to generate our formset was, in turn, generated with extra=1. This also happens to be the default, meaning we could have skipped specifying the extra argument and still gotten the same result.

As a final exercise in the shell, let's create a formset that has both user input data and initial data.

# shell
treasure_formset_initial_data = [
    {
    'name': 'Toyota AE86',
    'estimated_price': '3000',
    },
    {
    'name': 'Tofu',
    'estimated_price': '5',
    }
]
treasure_formset_userdata = {
    'form-TOTAL_FORMS': '2',
    'form-INITIAL_FORMS': '2',
    'form-MIN_NUM_FORMS': '0',
    'form-MAX_NUM_FORMS': '1000',
    'form-0-name': 'Fiat Hatchback',
    'form-0-estimated_price': '100',
    'form-1-name': 'Pasta',
    'form-1-estimated_price': '2',
}
TreasureFormSet = formset_factory(TreasureForm, extra=0)
treasure_formset = TreasureFormSet(treasure_formset_userdata, initial=treasure_formset_initial_data)
treasure_formset.has_changed() # True
treasure_formset.is_valid() # True
treasure_formset.cleaned_data 
# output: [{'name': 'Fiat Hatchback', 'estimated_price': 100}, {'name': 'Pasta', 'estimated_price': 2}]

Note that, even though the user input data had keys like 'form-0-name', the formset's cleaned data actually consists of a list of dictionaries, where each dictionary has the original field names as keys: 'name' and 'estimated_price' in our case.

That's enough of shelling, now let's turn our attention back to our project.

Forms in a Django project

Make sure you've exited the shell.

# shell
quit()

App creation

First we need to create an app.

(explorers_project) bash-3.2$ python manage.py startapp treasures

Your directory structure should now look something like

.
├── Pipfile
├── Pipfile.lock
├── db.sqlite3
├── explorer_project
│   ├── __init__.py
│   ├── asgi.py
│   ├── settings.py
│   ├── urls.py
│   └── wsgi.py
├── manage.py
├── treasures
│   ├── __init__.py
│   ├── admin.py
│   ├── apps.py
│   ├── migrations
│   │   └── __init__.py
│   ├── models.py
│   ├── tests.py
│   └── views.py

Make sure to add the treasures app to your INSTALLED_APPS list in the project's settings.py file.

Adding our Form subclass

Let's create a forms.py file for the treasures app.

(explorers_project) bash-3.2$ touch treasures/forms.py

Now let's put our trusty TreasureForm in there.

# treasures/forms.py
from django import forms

class TreasureForm(forms.Form):
    name = forms.CharField(max_length=100)
    estimated_price = forms.IntegerField()

Adding a view

Now we can jump over to views.py and write a simple function-based view (FBV) that we will use to render a template with the form.

# treasures/views.py
from django.shortcuts import render

from .forms import TreasureForm

def treasureform_view(request):
    treasure_form = TreasureForm()
    context = {
        'treasure_form': treasure_form
    }
    return render(request, 'treasures/treasure_submit.html', context)

Adding a template

We need to add a template file 'treasure_submit.html', which we referenced in our view. First we'll create a 'templates' subdirectory in the 'treasures' directory.

(explorers_project) bash-3.2$ mkdir treasures/templates

And then we create a sub-subdirectory, a 'treasures' directory inside of the 'templates' subdirectory (hopefully you're already familiar with this pattern - if not, check out the links at the beginning of this tutorial)

(explorers_project) bash-3.2$ mkdir treasures/templates/treasures

Finally we create our 'treasure_submit.html' file in the sub-subdirectory 'treasures'.

(explorers_project) bash-3.2$ touch treasures/templates/treasures/treasure_submit.html

The project structure at this point:

.
├── Pipfile
├── Pipfile.lock
├── db.sqlite3
├── explorer_project
│   ├── __init__.py
│   ├── asgi.py
│   ├── settings.py
│   ├── urls.py
│   └── wsgi.py
├── manage.py
├── treasures
│   ├── __init__.py
│   ├── admin.py
│   ├── apps.py
│   ├── forms.py
│   ├── migrations
│   │   └── __init__.py
│   ├── models.py
│   ├── templates
│   │   └── treasures
│   │       └── treasure_submit.html
│   ├── tests.py
│   └── views.py

Let's add some HTML to our 'treasure_submit.html' file.

# treasures/templates/treasures/treasure_submit.html
<!doctype html>

<html lang="en">
<head>
  <meta charset="utf-8">
  <title>Treasure submission</title>
</head>

<body>
    <form method='POST'>{% csrf_token %}
      {{ treasure_form.as_p }}
      <input type='submit' value="Submit treasure">
    </form>
</body>
</html>

The body holds:

• A <form> element, which django form inputs must always be enclosed by, with its method set to "POST"
• A {% csrf_token %} template tag, which we always need to include when using
• A {{ form.as_p }} template tag, telling django where to put the HTML that form.as_p() generates
• An <input> element with type set to 'submit', to provide a submit button (since django doesn't generate that either)

None of this should be new to you. (otherwise, again, check the links at the beginning)

Hooking up the view to a URL path

In order to make the view do its magic, we need to make it accessible through a URL path. First, let's change our project-level 'urls.py' file so that requests to the site are rerouted to a list of url patterns in 'treasures/urls.py' (which we'll create shortly)

# explorer_project/urls.py
from django.contrib import admin
from django.urls import path, include

urlpatterns = [
    path('admin/', admin.site.urls),
    path('', include('treasures.urls')),
]

Now we create the treasures app's 'urls.py' file and hook up our view to a url pattern.

(explorers_project) bash-3.2$ touch treasures/urls.py

# treasures/urls.py
from django.urls import path

from .views import treasureform_view

urlpatterns = [
    path('', treasureform_view)
]

Runtime

We might as well do a migrate, though we don't really need to at this point and it doesn't do anything else than set up the standard user tables, et c. It's just so that django won't yell at us about how we might have forgotten to migrate.

(explorers_project) bash-3.2$ python manage.py migrate

With that out of the way:

(explorers_project) bash-3.2$ python manage.py runserver

Now we can visit our site (by default, at 'http://127.0.0.1:8000/'). You should see a form with 'Name' and 'Estimated price' inputs, as well as a submit button. If you submit data at the moment, nothing really happens, since we haven't told our view to do anything differently if there's a POST request. Let's change that.

# treasures/views.py
from django.shortcuts import render
from django.http import HttpResponse

from .forms import TreasureForm


def treasureform_view(request):
    initial_form_data = {
        'name': 'Rock',
        'estimated_price': 0,
    }
    if request.method=="GET":
        treasure_form = TreasureForm(initial=initial_form_data)
    elif request.method=="POST":
        treasure_form = TreasureForm(data=request.POST,
                                     initial=initial_form_data)
        if treasure_form.is_valid() and treasure_form.has_changed():
            name = treasure_form.cleaned_data['name']
            estimated_price = treasure_form.cleaned_data['estimated_price']
            message = f'Thanks for your submission of {name}, \
                        with an estimated price of {estimated_price}!'
            return HttpResponse(message)
    context = {
        'treasure_form': treasure_form
    }
    return render(request, 'treasures/treasure_submit.html', context)

What our view does now is

• Regardless of whether the HTTP method of the request is GET (when the user just asks for the page) or POST (when the user submits their data), a form will be generated with initial data
  • If there are no user input data (GET request), the initial data will be used to pre-populate the form
  • If there are user input data (POST request), the initial data will be used for comparison with the user input data during the treasure_form.has_changed() call
• If it's a GET request, an unbound form is generated. This is simply rendered as before, except now it has initial field values.
• If it's a POST request, a bound form is created.
  • If the form holds valid data, they are cleaned and put in the .cleaned_data attribute as a dictionary.
  • If the user input are valid and differ from the initial data, the input values are retrieved and put in the variables name and estimated_price. The values are put inside of a message, using an f-string. The message is then sent back to the user in an HTTP response.
  • If the user input data are not valid (have been tampered with), the bound form, which includes information in the .errors attribute, will be used to render the form back to the user again, including error messages.
  • If the user input data are valid, but the same as the initial data, the bound form will be used to render the form back to the user again. Since no error has been raised, and the bound data are the same as the initial data, it will appear to the user as if they were are just shown the same page again. Obviously this isn't great for usability, so you'd have to tweak things for an actual site.

Try python manage.py runserver again and go to the site, to see how the page works and how it reacts to different inputs.

Formsets in a Django project

Now that we've gotten things to work with a simple form, let's switch to using a formset instead. We need to import the formset_factory method, create our FormSet subclass (our factory), and generate a formset, switching out the vanilla form we used above.

# treasures/views.py
from django.shortcuts import render
from django.http import HttpResponse
from django.forms import formset_factory

from .forms import TreasureForm


def treasureform_view(request):
    TreasureFormSet = formset_factory(TreasureForm, extra=2)
    treasure_formset = TreasureFormSet()
    context = {
        'treasure_formset': treasure_formset
    }
    return render(request, 'treasures/treasure_submit.html', context)

We also need to update our 'treasure_submit.html' template. When we worked in the shell, we saw that there is a shorthand we could use here, but for the sake of clarity, let's be explicit about the management form and looping through our forms.

# treasures/templates/treasures/treasure_submit.html
<!doctype html>

<html lang="en">
<head>
  <meta charset="utf-8">
  <title>Treasure submission</title>
</head>

<body>
    <form method='POST'>{% csrf_token %}
      {{ treasure_formset.management_form.as_p }}
      {% for treasure_form in treasure_formset.forms %}
        <h2>Form number {{ forloop.counter0 }}</h2>
        {{ treasure_form.as_p }}
        <hr>
      {% endfor %}
      <input type='submit' value="Submit treasure">
    </form>
</body>
</html>

To make things crystal clear, we label the forms by using a forloop counter starting at 0, since formsets' forms are counted starting from 0, as you saw back in the shell. If you're confused by the forloop counter, you can take a look at the Django docs about the for loop template tag.

Try python manage.py runserver and going to the form page again. You should now see two sets of name/price inputs, identical except for the header that precedes each one. Now try looking at the page's HTML/source. You can do this using your browser's tools, e. g. Inspect Element in Firefox. What you'll notice is that in addition to the two visible sets of inputs, there is HTML code for five inputs with type="hidden". The first one is named "csrfmiddlewaretoken" and is related to our template's {% csrf_token %} tag. The rest of the inputs represent the management form and are, unsurprisingly, exactly the same kind of output as we got in the shell when running treasure_formset.management_form.as_p().

If we try to submit any data now, we're back to the same situation as we were in a bit earlier - nothing much happens, we're just rendered the same empty input fields again. Let's make our view a bit more fancy again.

# treasures/views.py
from django.shortcuts import render
from django.http import HttpResponse
from django.forms import formset_factory

from .forms import TreasureForm


def treasureform_view(request):
    TreasureFormSet = formset_factory(TreasureForm, extra=1)
    treasure_formset = TreasureFormSet()
    initial_formset_data = [
        {
        'name': 'Rock',
        'estimated_price': 0,
         },
         {
         'name': 'Rock',
         'estimated_price': 0,
         },
    ]
    if request.method=="GET":
        treasure_formset = TreasureFormSet(initial=initial_formset_data)
    elif request.method=="POST":
        treasure_formset = TreasureFormSet(data=request.POST,
                                           initial=initial_formset_data)
        if treasure_formset.is_valid() and treasure_formset.has_changed():
            name0 = treasure_formset.cleaned_data[0]['name']
            estimated_price0 = treasure_formset.cleaned_data[0]['estimated_price']
            name1 = treasure_formset.cleaned_data[1]['name']
            estimated_price1 = treasure_formset.cleaned_data[1]['estimated_price']
            name2 = treasure_formset.cleaned_data[2]['name']
            estimated_price2 = treasure_formset.cleaned_data[2]['estimated_price']
            message = f'Thanks for your submission of {name0}, \
                        with an estimated price of {estimated_price0}! \
                        Thank you also for your submission of {name1}, \
                        with an estimated price of {estimated_price1}! \
                        And finally, thanks for your submission of {name2}, \
                        with an estimated price of {estimated_price2}!'
            return HttpResponse(message)
    context = {
        'treasure_formset': treasure_formset
    }
    return render(request, 'treasures/treasure_submit.html', context)

Let's once again go through what happens

• Whether it's a POST or GET request, a formset will be generated with initial data for two forms, and an "extra" empty form
  • If it's a GET request, initial data are used to pre-populate the formset's forms and affect the management form's tallies of total and initial number of forms
  • If it's a POST request, the initial data will be used for comparison (across all three forms) with the user input data during the treasure_formset.has_changed() call
• If it's a GET request, an unbound formset is generated.
• If it's a POST request, a bound formset is created.
  • If the formset holds valid data, they are cleaned and put in the .cleaned_data attribute as a list of dictionaries.
  • If the user input are valid and differ from the initial data, the input values are retrieved and put in the variables name/price variables. The values are put inside of a message which is sent to the user.
  • If the user input data aren't, the bound formset, which includes information in its .errors attribute, will be used to render the forms back to the user again, including error messages.
  • If the user input data are valid, but the same as the initial data, the bound form will be used to render the form back to the user again.

Run python manage.py runserver and have a look, check out the HTML/source like before and try submitting different kinds of input to see what happens.

Phew

That's it for now! In future tutorials building on this one, we'll look at:

• using formsets to save data to the database
• changing what "blueprint" the formset_factory function is to use for building formset classes
• dynamically adding/removing forms on a page

Please feel free to contact me at datalouvre@gmail.com if you have any feedback or questions. You can also commit PR's or raise issues on the series' github repo.